2024-04-28T12:19:19.430 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-28T12:19:19.430 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} --verbose --name rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878 -- /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878/orig.config.yaml 2024-04-28T12:19:19.476 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-28T12:19:19.583 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878 branch: wip-rishabh-testing-20240426.111959 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} email: null first_in_suite: false job_id: '7676878' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Reduced data availability - Degraded data redundancy sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNGKrQx3beiFGGGKr1IgBUo3gSRENd3cGQgZpEJ851txl5x6woWTKky3Fo7G49I7uRXQviEpZe6QxuTI98CKcvw= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAKcrhOvz6kpvYZgifp4WCPpwgGaA25tfLv08kMoYPpogHbOyuqFYVN/ZfdWJX+EUTPcB4G/dIKH8aVQiM70axQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_acls teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-28_11:41:23 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-28T12:19:19.583 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-28T12:19:19.584 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-28T12:19:19.584 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-28T12:19:19.585 INFO:teuthology.task.internal:Checking packages... 2024-04-28T12:19:19.608 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-28T12:19:19.608 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-28T12:19:19.608 INFO:teuthology.packaging:ref: None 2024-04-28T12:19:19.609 INFO:teuthology.packaging:tag: None 2024-04-28T12:19:19.609 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-28T12:19:19.609 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:19:19.609 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-28T12:19:19.753 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-28T12:19:19.755 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-28T12:19:19.764 INFO:teuthology.task.internal:no buildpackages task found 2024-04-28T12:19:19.764 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-28T12:19:19.799 INFO:teuthology.task.internal:Saving configuration 2024-04-28T12:19:19.815 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-28T12:19:19.820 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-28T12:19:19.845 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 12:12:14.727757', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNGKrQx3beiFGGGKr1IgBUo3gSRENd3cGQgZpEJ851txl5x6woWTKky3Fo7G49I7uRXQviEpZe6QxuTI98CKcvw='} 2024-04-28T12:19:19.866 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 12:12:14.725859', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAKcrhOvz6kpvYZgifp4WCPpwgGaA25tfLv08kMoYPpogHbOyuqFYVN/ZfdWJX+EUTPcB4G/dIKH8aVQiM70axQ='} 2024-04-28T12:19:19.866 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-28T12:19:19.872 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-28T12:19:19.872 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-28T12:19:19.872 INFO:teuthology.run_tasks:Running task console_log... 2024-04-28T12:19:19.942 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9d2f053160>, signals=[15]) 2024-04-28T12:19:19.943 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-28T12:19:19.948 INFO:teuthology.task.internal:Opening connections... 2024-04-28T12:19:19.948 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:19:19.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:19:20.033 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:19:20.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:19:20.110 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-28T12:19:20.116 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-04-28T12:19:20.134 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-04-28T12:19:20.134 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-04-28T12:19:20.191 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2024-04-28T12:19:20.191 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9" 2024-04-28T12:19:20.191 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2024-04-28T12:19:20.191 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2024-04-28T12:19:20.191 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9" 2024-04-28T12:19:20.191 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9" 2024-04-28T12:19:20.191 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-28T12:19:20.191 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-04-28T12:19:20.191 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon" 2024-04-28T12:19:20.191 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-28T12:19:20.192 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2024-04-28T12:19:20.192 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-28T12:19:20.192 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-28T12:19:20.192 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-28T12:19:20.192 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-04-28T12:19:20.215 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-04-28T12:19:20.233 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-04-28T12:19:20.234 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-04-28T12:19:20.290 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-04-28T12:19:20.290 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-04-28T12:19:20.290 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-04-28T12:19:20.290 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-04-28T12:19:20.290 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-04-28T12:19:20.290 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-04-28T12:19:20.290 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-28T12:19:20.290 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-04-28T12:19:20.290 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-04-28T12:19:20.290 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-28T12:19:20.290 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-04-28T12:19:20.290 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-28T12:19:20.291 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-28T12:19:20.291 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-28T12:19:20.291 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-04-28T12:19:20.313 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-28T12:19:20.322 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-28T12:19:20.327 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-28T12:19:20.328 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-04-28T12:19:20.330 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-04-28T12:19:20.346 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-28T12:19:20.351 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-28T12:19:20.351 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-04-28T12:19:20.386 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-04-28T12:19:20.484 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-28T12:19:20.534 INFO:teuthology.run_tasks:Running task kernel... 2024-04-28T12:19:20.550 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-28T12:19:20.550 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-28T12:19:20.550 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-28T12:19:20.551 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-28T12:19:20.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:19:20.568 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T12:19:20.568 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-28T12:19:20.568 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-04-28T12:19:20.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:19:20.570 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T12:19:20.570 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-28T12:19:20.570 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-04-28T12:19:20.625 INFO:teuthology.orchestra.run.smithi131.stdout:5.14.0-437.el9.x86_64 2024-04-28T12:19:20.625 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T12:19:20.625 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-28T12:19:20.625 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T12:19:20.629 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-437.el9.x86_64 2024-04-28T12:19:20.629 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T12:19:20.629 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-28T12:19:20.629 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T12:19:20.771 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum upgrade -y linux-firmware 2024-04-28T12:19:20.944 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum upgrade -y linux-firmware 2024-04-28T12:19:22.987 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-04-28T12:19:23.680 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.0 MB 00:01 2024-04-28T12:19:26.905 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 19 MB 00:02 2024-04-28T12:19:32.430 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-04-28T12:19:33.789 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 16 kB 00:00 2024-04-28T12:19:35.199 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-28T12:19:40.465 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-28T12:19:42.421 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-28T12:19:42.422 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-04-28T12:19:42.422 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-28T12:19:42.551 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-28T12:19:42.552 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T12:19:42.888 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-28T12:19:42.888 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-28T12:19:42.891 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi157.front.sepia.ceph.com to complete... 2024-04-28T12:19:44.676 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 976 kB/s | 19 MB 00:19 2024-04-28T12:19:49.787 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-04-28T12:19:51.461 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-04-28T12:19:53.970 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 21 MB 00:02 2024-04-28T12:19:59.073 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-28T12:20:01.049 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-04-28T12:20:01.050 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-04-28T12:20:01.050 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-04-28T12:20:01.160 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-28T12:20:01.161 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T12:20:01.503 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-28T12:20:01.503 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-28T12:20:01.506 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi131.front.sepia.ceph.com to complete... 2024-04-28T12:23:39.193 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28 12:23:39 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-28T12:23:39.194 INFO:teuthology.orchestra.run.smithi157.stderr:FINISHED --2024-04-28 12:23:39-- 2024-04-28T12:23:39.194 INFO:teuthology.orchestra.run.smithi157.stderr:Total wall clock time: 3m 56s 2024-04-28T12:23:39.194 INFO:teuthology.orchestra.run.smithi157.stderr:Downloaded: 1 files, 341M in 3m 56s (1.45 MB/s) 2024-04-28T12:23:39.243 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-28T12:23:39.243 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-28T12:23:41.364 INFO:teuthology.orchestra.run.smithi157.stdout:Verifying... ######################################## 2024-04-28T12:23:41.595 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing... ######################################## 2024-04-28T12:23:41.704 INFO:teuthology.orchestra.run.smithi157.stdout:Updating / installing... 2024-04-28T12:23:49.800 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-28T12:23:52.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28 12:23:52 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-28T12:23:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:FINISHED --2024-04-28 12:23:52-- 2024-04-28T12:23:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:Total wall clock time: 3m 51s 2024-04-28T12:23:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:Downloaded: 1 files, 341M in 3m 50s (1.48 MB/s) 2024-04-28T12:23:52.229 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-28T12:23:52.230 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-28T12:23:54.376 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying... ######################################## 2024-04-28T12:23:54.491 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing... ######################################## 2024-04-28T12:23:54.598 INFO:teuthology.orchestra.run.smithi131.stdout:Updating / installing... 2024-04-28T12:24:02.509 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-28T12:24:27.256 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-28T12:24:27.256 DEBUG:teuthology.orchestra.run.smithi157:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-28T12:24:27.348 INFO:teuthology.orchestra.run.smithi157.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:27.349 INFO:teuthology.orchestra.run.smithi157.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:27.349 INFO:teuthology.orchestra.run.smithi157.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:27.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-28T12:24:27.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-28T12:24:27.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-28T12:24:27.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-28T12:24:27.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-28T12:24:27.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-28T12:24:27.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-28T12:24:27.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-28T12:24:27.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-28T12:24:27.349 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-28T12:24:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-28T12:24:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-28T12:24:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-28T12:24:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-28T12:24:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-28T12:24:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-28T12:24:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-28T12:24:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-28T12:24:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-28T12:24:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-28T12:24:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-28T12:24:27.350 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-28T12:24:27.351 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-28T12:24:27.352 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-28T12:24:27.353 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-28T12:24:27.354 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-28T12:24:27.355 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-28T12:24:27.356 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-28T12:24:27.357 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-28T12:24:27.358 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-28T12:24:27.359 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-28T12:24:27.359 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-28T12:24:27.359 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-28T12:24:27.359 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-28T12:24:27.359 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-28T12:24:27.359 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-28T12:24:27.359 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-28T12:24:27.359 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-28T12:24:27.360 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-28T12:24:27.360 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-28T12:24:27.360 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-28T12:24:27.360 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-28T12:24:27.360 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-28T12:24:27.360 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-28T12:24:27.360 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-28T12:24:27.361 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-28T12:24:27.361 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-28T12:24:27.361 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-28T12:24:27.361 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-28T12:24:27.361 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-28T12:24:27.361 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-28T12:24:27.361 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-28T12:24:27.361 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-28T12:24:27.361 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-28T12:24:27.362 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-28T12:24:27.362 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-28T12:24:27.362 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-28T12:24:27.362 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-28T12:24:27.362 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-28T12:24:27.362 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-28T12:24:27.362 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-28T12:24:27.362 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-28T12:24:27.363 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-28T12:24:27.364 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-28T12:24:27.365 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-28T12:24:27.366 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-28T12:24:27.367 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-28T12:24:27.368 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-28T12:24:27.369 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-28T12:24:27.369 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-28T12:24:27.369 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-28T12:24:27.369 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-28T12:24:27.369 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-28T12:24:27.369 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-28T12:24:27.369 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-28T12:24:27.369 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-28T12:24:27.369 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-28T12:24:27.369 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-28T12:24:27.369 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-28T12:24:27.369 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-28T12:24:27.370 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-28T12:24:27.370 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-28T12:24:27.370 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-28T12:24:27.370 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-28T12:24:27.370 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-28T12:24:27.370 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-28T12:24:27.370 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-28T12:24:27.370 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-28T12:24:27.370 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-28T12:24:27.370 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-28T12:24:27.371 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-28T12:24:27.372 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-28T12:24:27.373 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-28T12:24:27.374 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-28T12:24:27.374 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-28T12:24:27.374 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-28T12:24:27.374 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-28T12:24:27.374 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-28T12:24:27.374 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-28T12:24:27.374 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-28T12:24:27.374 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-28T12:24:27.374 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-28T12:24:27.375 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-28T12:24:27.375 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-28T12:24:27.375 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-28T12:24:27.375 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-28T12:24:27.375 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-28T12:24:27.375 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-28T12:24:27.375 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-28T12:24:27.375 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-28T12:24:27.376 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-28T12:24:27.376 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-28T12:24:27.376 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-28T12:24:27.376 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-28T12:24:27.376 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-28T12:24:27.376 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-28T12:24:27.376 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-28T12:24:27.377 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-28T12:24:27.377 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-28T12:24:27.377 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-28T12:24:27.377 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-28T12:24:27.377 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-28T12:24:27.377 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-28T12:24:27.378 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-28T12:24:27.378 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-28T12:24:27.378 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-28T12:24:27.378 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-28T12:24:27.378 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-28T12:24:27.378 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-28T12:24:27.378 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-28T12:24:27.379 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-28T12:24:27.379 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-28T12:24:27.379 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-28T12:24:27.379 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-28T12:24:27.379 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-28T12:24:27.379 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-28T12:24:27.380 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-28T12:24:27.380 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-28T12:24:27.380 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-28T12:24:27.380 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-28T12:24:27.380 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-28T12:24:27.380 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-28T12:24:27.380 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-28T12:24:27.380 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-28T12:24:27.381 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-28T12:24:27.381 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-28T12:24:27.381 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-28T12:24:27.381 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-28T12:24:27.381 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-28T12:24:27.381 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-28T12:24:27.381 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-28T12:24:27.381 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-28T12:24:27.382 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-28T12:24:27.382 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-28T12:24:27.382 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-28T12:24:27.382 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-28T12:24:27.382 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-28T12:24:27.382 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-28T12:24:27.383 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-28T12:24:27.383 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-28T12:24:27.383 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-28T12:24:27.383 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-28T12:24:27.383 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-28T12:24:27.383 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-28T12:24:27.384 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-28T12:24:27.384 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-28T12:24:27.384 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-28T12:24:27.384 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-28T12:24:27.384 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-28T12:24:27.384 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-28T12:24:27.385 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-28T12:24:27.385 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-28T12:24:27.385 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-28T12:24:27.385 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-28T12:24:27.385 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-28T12:24:27.385 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-28T12:24:27.385 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-28T12:24:27.385 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-28T12:24:27.385 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-28T12:24:27.385 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-28T12:24:27.385 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-28T12:24:27.386 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-28T12:24:27.386 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-28T12:24:27.386 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-28T12:24:27.386 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-28T12:24:27.386 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-28T12:24:27.386 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-28T12:24:27.386 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-28T12:24:27.386 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-28T12:24:27.386 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-28T12:24:27.387 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-28T12:24:27.387 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-28T12:24:27.387 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-28T12:24:27.387 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-28T12:24:27.387 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-28T12:24:27.387 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-28T12:24:27.387 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-28T12:24:27.387 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-28T12:24:27.387 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-28T12:24:27.388 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-28T12:24:27.389 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-28T12:24:27.390 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-28T12:24:27.390 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-28T12:24:27.390 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-28T12:24:27.390 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-28T12:24:27.390 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-28T12:24:27.390 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-28T12:24:27.390 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-28T12:24:27.390 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-28T12:24:27.390 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-28T12:24:27.390 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-28T12:24:27.390 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-28T12:24:27.391 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-28T12:24:27.392 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-28T12:24:27.392 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-28T12:24:27.392 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-28T12:24:27.392 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-28T12:24:27.392 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-28T12:24:27.392 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-28T12:24:27.392 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-28T12:24:27.392 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-28T12:24:27.392 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-28T12:24:27.392 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-28T12:24:27.392 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-28T12:24:27.393 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-28T12:24:27.394 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-28T12:24:27.394 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-28T12:24:27.394 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-28T12:24:27.394 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-28T12:24:27.394 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-28T12:24:27.394 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-28T12:24:27.394 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-28T12:24:27.394 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-28T12:24:27.394 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-28T12:24:27.394 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-28T12:24:27.394 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-28T12:24:27.394 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-28T12:24:27.395 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-28T12:24:27.395 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-28T12:24:27.395 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-28T12:24:27.395 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-28T12:24:27.395 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-28T12:24:27.395 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-28T12:24:27.396 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-28T12:24:27.396 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-28T12:24:27.396 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-28T12:24:27.396 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-28T12:24:27.396 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-28T12:24:27.396 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-28T12:24:27.396 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-28T12:24:27.396 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-28T12:24:27.397 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-28T12:24:27.397 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-28T12:24:27.397 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-28T12:24:27.397 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-28T12:24:27.397 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-28T12:24:27.397 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-28T12:24:27.397 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-28T12:24:27.397 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-28T12:24:27.397 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-28T12:24:27.398 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-28T12:24:27.399 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-28T12:24:27.400 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-28T12:24:27.400 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-28T12:24:27.400 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-28T12:24:27.400 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-28T12:24:27.400 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-28T12:24:27.400 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-28T12:24:27.400 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-28T12:24:27.400 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-28T12:24:27.400 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-28T12:24:27.401 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-28T12:24:27.401 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-28T12:24:27.401 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-28T12:24:27.401 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-28T12:24:27.401 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-28T12:24:27.401 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-28T12:24:27.401 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-28T12:24:27.402 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-28T12:24:27.403 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-28T12:24:27.404 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-28T12:24:27.405 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-28T12:24:27.406 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-28T12:24:27.407 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-28T12:24:27.408 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-28T12:24:27.409 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-28T12:24:27.409 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-28T12:24:27.409 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-28T12:24:27.409 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-28T12:24:27.409 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-28T12:24:27.409 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-28T12:24:27.409 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-28T12:24:27.409 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-28T12:24:27.410 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-28T12:24:27.410 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-28T12:24:27.410 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-28T12:24:27.410 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-28T12:24:27.410 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-28T12:24:27.410 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-28T12:24:27.410 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-28T12:24:27.410 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-28T12:24:27.410 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-28T12:24:27.410 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-28T12:24:27.411 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-28T12:24:27.411 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-28T12:24:27.411 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-28T12:24:27.411 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-28T12:24:27.411 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-28T12:24:27.411 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-28T12:24:27.411 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-28T12:24:27.411 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-28T12:24:27.412 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-28T12:24:27.412 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-28T12:24:27.412 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-28T12:24:27.412 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-28T12:24:27.412 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-28T12:24:27.412 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-28T12:24:27.413 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-28T12:24:27.414 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-28T12:24:27.414 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-28T12:24:27.414 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-28T12:24:27.414 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-28T12:24:27.414 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-28T12:24:27.414 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-28T12:24:27.414 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-28T12:24:27.414 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-28T12:24:27.414 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-28T12:24:27.414 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-28T12:24:27.415 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-28T12:24:27.415 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-28T12:24:27.415 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-28T12:24:27.415 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-28T12:24:27.415 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-28T12:24:27.415 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-28T12:24:27.415 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-28T12:24:27.415 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-28T12:24:27.416 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-28T12:24:27.416 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-28T12:24:27.416 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-28T12:24:27.416 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-28T12:24:27.416 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-28T12:24:27.416 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-28T12:24:27.416 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-28T12:24:27.416 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-28T12:24:27.417 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-28T12:24:27.417 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-28T12:24:27.417 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-28T12:24:27.417 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-28T12:24:27.417 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-28T12:24:27.417 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-28T12:24:27.417 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-28T12:24:27.418 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-28T12:24:27.418 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-28T12:24:27.418 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-28T12:24:27.418 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-28T12:24:27.418 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-28T12:24:27.418 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-28T12:24:27.419 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-28T12:24:27.419 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-28T12:24:27.419 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-28T12:24:27.419 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-28T12:24:27.419 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-28T12:24:27.419 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-28T12:24:27.419 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-28T12:24:27.420 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-28T12:24:27.420 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-28T12:24:27.420 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-28T12:24:27.420 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-28T12:24:27.420 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-28T12:24:27.420 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-28T12:24:27.420 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-28T12:24:27.421 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-28T12:24:27.421 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-28T12:24:27.421 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-28T12:24:27.421 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-28T12:24:27.421 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-28T12:24:27.421 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-28T12:24:27.421 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-28T12:24:27.422 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-28T12:24:27.422 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-28T12:24:27.422 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-28T12:24:27.422 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-28T12:24:27.422 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-28T12:24:27.422 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-28T12:24:27.422 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-28T12:24:27.422 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-28T12:24:27.422 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-28T12:24:27.422 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-28T12:24:27.422 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-28T12:24:27.423 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-28T12:24:27.424 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-28T12:24:27.425 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-28T12:24:27.426 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-28T12:24:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-28T12:24:27.428 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-28T12:24:27.429 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-28T12:24:27.430 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-28T12:24:27.431 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-28T12:24:27.432 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-28T12:24:27.433 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-28T12:24:27.434 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-28T12:24:27.435 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-28T12:24:27.435 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-28T12:24:27.435 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-28T12:24:27.435 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-28T12:24:27.435 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-28T12:24:27.435 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-28T12:24:27.435 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-28T12:24:27.435 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-28T12:24:27.435 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-28T12:24:27.435 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-28T12:24:27.435 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-28T12:24:27.435 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-28T12:24:27.436 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-28T12:24:27.437 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-28T12:24:27.438 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-28T12:24:27.439 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-28T12:24:27.440 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-28T12:24:27.441 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-28T12:24:27.442 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-28T12:24:27.443 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-28T12:24:27.444 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-28T12:24:27.445 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-28T12:24:27.446 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-28T12:24:27.447 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-28T12:24:27.448 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-28T12:24:27.449 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-28T12:24:27.450 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-28T12:24:27.451 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-28T12:24:27.452 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-28T12:24:27.453 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-28T12:24:27.454 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-28T12:24:27.455 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-28T12:24:27.456 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-28T12:24:27.457 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-28T12:24:27.458 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-28T12:24:27.459 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-28T12:24:27.459 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-28T12:24:27.459 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-28T12:24:27.459 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-28T12:24:27.459 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-28T12:24:27.459 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-28T12:24:27.459 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-28T12:24:27.459 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-28T12:24:27.459 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-28T12:24:27.459 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-28T12:24:27.460 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-28T12:24:27.461 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-28T12:24:27.462 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-28T12:24:27.463 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-28T12:24:27.464 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-28T12:24:27.465 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-28T12:24:27.466 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-28T12:24:27.467 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-28T12:24:27.468 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-28T12:24:27.469 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-28T12:24:27.470 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-28T12:24:27.471 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-28T12:24:27.472 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-28T12:24:27.473 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-28T12:24:27.474 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-28T12:24:27.475 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-28T12:24:27.476 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-28T12:24:27.477 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-28T12:24:27.478 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-28T12:24:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-28T12:24:27.480 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-28T12:24:27.481 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-28T12:24:27.482 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-28T12:24:27.483 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-28T12:24:27.484 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-28T12:24:27.485 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-28T12:24:27.486 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-28T12:24:27.487 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-28T12:24:27.488 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-28T12:24:27.489 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-28T12:24:27.490 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-28T12:24:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-28T12:24:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-28T12:24:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-28T12:24:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-28T12:24:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-28T12:24:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-28T12:24:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-28T12:24:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-28T12:24:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-28T12:24:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-28T12:24:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-28T12:24:27.491 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-28T12:24:27.492 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-28T12:24:27.493 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-28T12:24:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-28T12:24:27.495 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-28T12:24:27.496 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-28T12:24:27.497 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-28T12:24:27.498 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-28T12:24:27.499 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-28T12:24:27.500 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-28T12:24:27.501 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-28T12:24:27.502 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-28T12:24:27.503 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-28T12:24:27.503 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-28T12:24:27.503 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-28T12:24:27.503 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-28T12:24:27.503 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-28T12:24:27.503 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-28T12:24:27.503 INFO:teuthology.orchestra.run.smithi157.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-28T12:24:27.507 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:27.507 DEBUG:teuthology.orchestra.run.smithi157:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-28T12:24:27.534 INFO:teuthology.orchestra.run.smithi157.stdout:postinstall scriptlet (using /bin/sh): 2024-04-28T12:24:27.535 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-28T12:24:27.535 INFO:teuthology.orchestra.run.smithi157.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-28T12:24:27.535 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2024-04-28T12:24:27.535 INFO:teuthology.orchestra.run.smithi157.stdout:for file in vmlinuz System.map config; do 2024-04-28T12:24:27.535 INFO:teuthology.orchestra.run.smithi157.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-28T12:24:27.535 INFO:teuthology.orchestra.run.smithi157.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-28T12:24:27.535 INFO:teuthology.orchestra.run.smithi157.stdout: fi 2024-04-28T12:24:27.535 INFO:teuthology.orchestra.run.smithi157.stdout:done 2024-04-28T12:24:27.535 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-28T12:24:27.535 INFO:teuthology.orchestra.run.smithi157.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-28T12:24:27.535 INFO:teuthology.orchestra.run.smithi157.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-28T12:24:27.536 INFO:teuthology.orchestra.run.smithi157.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-28T12:24:27.536 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2024-04-28T12:24:27.536 INFO:teuthology.orchestra.run.smithi157.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-28T12:24:27.536 INFO:teuthology.orchestra.run.smithi157.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:27.536 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2024-04-28T12:24:27.536 INFO:teuthology.orchestra.run.smithi157.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-28T12:24:27.536 INFO:teuthology.orchestra.run.smithi157.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-28T12:24:27.536 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:27.536 INFO:teuthology.orchestra.run.smithi157.stdout:fi 2024-04-28T12:24:27.536 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -qi grub2-tools 2024-04-28T12:24:27.615 INFO:teuthology.orchestra.run.smithi157.stdout:Name : grub2-tools 2024-04-28T12:24:27.615 INFO:teuthology.orchestra.run.smithi157.stdout:Epoch : 1 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Version : 2.06 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Release : 68.el9 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: x86_64 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Group : Unspecified 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Size : 8264254 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:License : GPLv3+ 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Packager : builder@centos.org 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Vendor : CentOS 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-28T12:24:27.616 INFO:teuthology.orchestra.run.smithi157.stdout:Summary : Support tools for GRUB. 2024-04-28T12:24:27.617 INFO:teuthology.orchestra.run.smithi157.stdout:Description : 2024-04-28T12:24:27.617 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:24:27.617 INFO:teuthology.orchestra.run.smithi157.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-28T12:24:27.617 INFO:teuthology.orchestra.run.smithi157.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-28T12:24:27.617 INFO:teuthology.orchestra.run.smithi157.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-28T12:24:27.617 INFO:teuthology.orchestra.run.smithi157.stdout:hardware devices. 2024-04-28T12:24:27.617 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:24:27.617 INFO:teuthology.orchestra.run.smithi157.stdout:This subpackage provides tools for support of all platforms. 2024-04-28T12:24:27.618 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-28T12:24:27.618 INFO:teuthology.task.kernel:Updating grub on smithi157 to boot 6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:27.618 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-28T12:24:28.189 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-04-28T12:24:29.648 INFO:teuthology.orchestra.run.smithi157.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-28T12:24:29.665 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-04-28T12:24:29.668 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-04-28T12:24:29.683 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.XW844P5lZe 2024-04-28T12:24:29.683 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XW844P5lZe 2024-04-28T12:24:29.749 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.XW844P5lZe 2024-04-28T12:24:29.892 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.XW844P5lZe is 6KB 2024-04-28T12:24:29.940 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.XW844P5lZe 2024-04-28T12:24:29.954 DEBUG:teuthology.orchestra.run.smithi157:> sudo /bin/ls /boot/loader/entries || true 2024-04-28T12:24:30.016 INFO:teuthology.orchestra.run.smithi157.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-28T12:24:30.016 INFO:teuthology.orchestra.run.smithi157.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-28T12:24:30.016 INFO:teuthology.orchestra.run.smithi157.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-28T12:24:30.016 INFO:teuthology.orchestra.run.smithi157.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-28T12:24:30.018 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:30.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2024-04-28T12:24:40.757 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi131.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-28T12:24:40.757 DEBUG:teuthology.orchestra.run.smithi131:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-28T12:24:40.835 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:40.835 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-28T12:24:40.836 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-28T12:24:40.837 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-28T12:24:40.838 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-28T12:24:40.839 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-28T12:24:40.840 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-28T12:24:40.841 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-28T12:24:40.842 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-28T12:24:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-28T12:24:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-28T12:24:40.845 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-28T12:24:40.846 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-28T12:24:40.847 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-28T12:24:40.848 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-28T12:24:40.849 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-28T12:24:40.850 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-28T12:24:40.851 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-28T12:24:40.852 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-28T12:24:40.853 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-28T12:24:40.854 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-28T12:24:40.855 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-28T12:24:40.856 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-28T12:24:40.857 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-28T12:24:40.858 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-28T12:24:40.859 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-28T12:24:40.860 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-28T12:24:40.861 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-28T12:24:40.862 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-28T12:24:40.863 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-28T12:24:40.864 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-28T12:24:40.865 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-28T12:24:40.866 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-28T12:24:40.867 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-28T12:24:40.868 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-28T12:24:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-28T12:24:40.870 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-28T12:24:40.871 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-28T12:24:40.872 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-28T12:24:40.873 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-28T12:24:40.874 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-28T12:24:40.875 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-28T12:24:40.876 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-28T12:24:40.877 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-28T12:24:40.878 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-28T12:24:40.879 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-28T12:24:40.880 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-28T12:24:40.881 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-28T12:24:40.881 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-28T12:24:40.881 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-28T12:24:40.881 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-28T12:24:40.881 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-28T12:24:40.881 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-28T12:24:40.881 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-28T12:24:40.881 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-28T12:24:40.882 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-28T12:24:40.883 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-28T12:24:40.884 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-28T12:24:40.885 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-28T12:24:40.886 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-28T12:24:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-28T12:24:40.888 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-28T12:24:40.889 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-28T12:24:40.890 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-28T12:24:40.891 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-28T12:24:40.892 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-28T12:24:40.893 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-28T12:24:40.894 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-28T12:24:40.895 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-28T12:24:40.896 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-28T12:24:40.897 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-28T12:24:40.898 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-28T12:24:40.899 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-28T12:24:40.900 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-28T12:24:40.901 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-28T12:24:40.902 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-28T12:24:40.903 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-28T12:24:40.904 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-28T12:24:40.905 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-28T12:24:40.906 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-28T12:24:40.907 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-28T12:24:40.908 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-28T12:24:40.909 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-28T12:24:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-28T12:24:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-28T12:24:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-28T12:24:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-28T12:24:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-28T12:24:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-28T12:24:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-28T12:24:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-28T12:24:40.910 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-28T12:24:40.911 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-28T12:24:40.912 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-28T12:24:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-28T12:24:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-28T12:24:40.915 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-28T12:24:40.916 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-28T12:24:40.917 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-28T12:24:40.918 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-28T12:24:40.919 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-28T12:24:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-28T12:24:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-28T12:24:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-28T12:24:40.923 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-28T12:24:40.924 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-28T12:24:40.925 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-28T12:24:40.926 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-28T12:24:40.927 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-28T12:24:40.928 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-28T12:24:40.929 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-28T12:24:40.930 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-28T12:24:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-28T12:24:40.932 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-28T12:24:40.933 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-28T12:24:40.934 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-28T12:24:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-28T12:24:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-28T12:24:40.937 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-28T12:24:40.938 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-28T12:24:40.939 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-28T12:24:40.940 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-28T12:24:40.941 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-28T12:24:40.942 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-28T12:24:40.943 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-28T12:24:40.944 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-28T12:24:40.945 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-28T12:24:40.946 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-28T12:24:40.947 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-28T12:24:40.948 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-28T12:24:40.949 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-28T12:24:40.950 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-28T12:24:40.951 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-28T12:24:40.952 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-28T12:24:40.953 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-28T12:24:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-28T12:24:40.955 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-28T12:24:40.956 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-28T12:24:40.957 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-28T12:24:40.958 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-28T12:24:40.959 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-28T12:24:40.960 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-28T12:24:40.961 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-28T12:24:40.962 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-28T12:24:40.963 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-28T12:24:40.964 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-28T12:24:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-28T12:24:40.966 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-28T12:24:40.967 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-28T12:24:40.968 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-28T12:24:40.969 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-28T12:24:40.970 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-28T12:24:40.971 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-28T12:24:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-28T12:24:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-28T12:24:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-28T12:24:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-28T12:24:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-28T12:24:40.972 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-28T12:24:40.975 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:40.976 DEBUG:teuthology.orchestra.run.smithi131:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-28T12:24:40.997 INFO:teuthology.orchestra.run.smithi131.stdout:postinstall scriptlet (using /bin/sh): 2024-04-28T12:24:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-28T12:24:40.998 INFO:teuthology.orchestra.run.smithi131.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-28T12:24:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-04-28T12:24:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:for file in vmlinuz System.map config; do 2024-04-28T12:24:40.998 INFO:teuthology.orchestra.run.smithi131.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-28T12:24:40.998 INFO:teuthology.orchestra.run.smithi131.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-28T12:24:40.998 INFO:teuthology.orchestra.run.smithi131.stdout: fi 2024-04-28T12:24:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:done 2024-04-28T12:24:40.998 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-28T12:24:40.998 INFO:teuthology.orchestra.run.smithi131.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-28T12:24:40.998 INFO:teuthology.orchestra.run.smithi131.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-28T12:24:40.998 INFO:teuthology.orchestra.run.smithi131.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-28T12:24:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-04-28T12:24:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-28T12:24:40.999 INFO:teuthology.orchestra.run.smithi131.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-04-28T12:24:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-28T12:24:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-28T12:24:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:40.999 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-04-28T12:24:41.000 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -qi grub2-tools 2024-04-28T12:24:41.072 INFO:teuthology.orchestra.run.smithi131.stdout:Name : grub2-tools 2024-04-28T12:24:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:Epoch : 1 2024-04-28T12:24:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:Version : 2.06 2024-04-28T12:24:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:Release : 68.el9 2024-04-28T12:24:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: x86_64 2024-04-28T12:24:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-28T12:24:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:Group : Unspecified 2024-04-28T12:24:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:Size : 8264254 2024-04-28T12:24:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:License : GPLv3+ 2024-04-28T12:24:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-28T12:24:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-28T12:24:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-28T12:24:41.073 INFO:teuthology.orchestra.run.smithi131.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-28T12:24:41.074 INFO:teuthology.orchestra.run.smithi131.stdout:Packager : builder@centos.org 2024-04-28T12:24:41.074 INFO:teuthology.orchestra.run.smithi131.stdout:Vendor : CentOS 2024-04-28T12:24:41.074 INFO:teuthology.orchestra.run.smithi131.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-28T12:24:41.074 INFO:teuthology.orchestra.run.smithi131.stdout:Summary : Support tools for GRUB. 2024-04-28T12:24:41.074 INFO:teuthology.orchestra.run.smithi131.stdout:Description : 2024-04-28T12:24:41.074 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:24:41.074 INFO:teuthology.orchestra.run.smithi131.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-28T12:24:41.074 INFO:teuthology.orchestra.run.smithi131.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-28T12:24:41.074 INFO:teuthology.orchestra.run.smithi131.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-28T12:24:41.074 INFO:teuthology.orchestra.run.smithi131.stdout:hardware devices. 2024-04-28T12:24:41.074 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:24:41.074 INFO:teuthology.orchestra.run.smithi131.stdout:This subpackage provides tools for support of all platforms. 2024-04-28T12:24:41.075 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-28T12:24:41.075 INFO:teuthology.task.kernel:Updating grub on smithi131 to boot 6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:41.075 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-28T12:24:41.607 INFO:teuthology.orchestra.run.smithi131.stderr:Generating grub configuration file ... 2024-04-28T12:24:42.988 INFO:teuthology.orchestra.run.smithi131.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-28T12:24:43.020 INFO:teuthology.orchestra.run.smithi131.stderr:done 2024-04-28T12:24:43.022 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-04-28T12:24:43.037 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.GWyG9VAmQW 2024-04-28T12:24:43.037 DEBUG:teuthology.orchestra.run.smithi131:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.GWyG9VAmQW 2024-04-28T12:24:43.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.GWyG9VAmQW 2024-04-28T12:24:43.242 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.GWyG9VAmQW is 6KB 2024-04-28T12:24:43.290 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.GWyG9VAmQW 2024-04-28T12:24:43.304 DEBUG:teuthology.orchestra.run.smithi131:> sudo /bin/ls /boot/loader/entries || true 2024-04-28T12:24:43.371 INFO:teuthology.orchestra.run.smithi131.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-28T12:24:43.372 INFO:teuthology.orchestra.run.smithi131.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-28T12:24:43.372 INFO:teuthology.orchestra.run.smithi131.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-28T12:24:43.372 INFO:teuthology.orchestra.run.smithi131.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-28T12:24:43.373 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:43.532 DEBUG:teuthology.orchestra.run.smithi131:> sudo shutdown -r now 2024-04-28T12:25:00.190 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-28T12:25:00.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-28T12:25:00.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:25:13.550 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-28T12:25:13.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-04-28T12:25:13.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:25:34.751 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-04-28T12:25:43.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-28T12:25:43.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:25:46.814 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-04-28T12:25:58.826 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-28T12:25:58.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:26:13.569 DEBUG:teuthology.orchestra.remote:timed out 2024-04-28T12:26:17.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-04-28T12:26:22.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-04-28T12:26:22.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:26:23.029 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-04-28T12:26:23.562 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-04-28T12:26:23.562 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-28T12:26:23.562 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-04-28T12:26:23.601 INFO:teuthology.orchestra.run.smithi131.stdout:6.9.0-rc3-g806ec36b2112 2024-04-28T12:26:23.601 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-28T12:26:23.601 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-28T12:26:23.601 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-28T12:26:24.602 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-28T12:26:24.602 DEBUG:teuthology.orchestra.run.smithi131:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-28T12:26:24.737 INFO:teuthology.orchestra.run.smithi131.stdout:ttyS1 2024-04-28T12:26:24.798 DEBUG:teuthology.parallel:result is None 2024-04-28T12:26:32.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-28T12:26:32.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:26:35.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-04-28T12:26:53.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-28T12:26:53.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:26:53.731 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-04-28T12:26:54.281 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-28T12:26:54.281 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-28T12:26:54.281 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-04-28T12:26:54.319 INFO:teuthology.orchestra.run.smithi157.stdout:6.9.0-rc3-g806ec36b2112 2024-04-28T12:26:54.320 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-28T12:26:54.320 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-28T12:26:54.320 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-28T12:26:55.321 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-28T12:26:55.321 DEBUG:teuthology.orchestra.run.smithi157:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-28T12:26:55.650 INFO:teuthology.orchestra.run.smithi157.stdout:ttyS1 2024-04-28T12:26:55.761 DEBUG:teuthology.parallel:result is None 2024-04-28T12:26:55.762 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-28T12:26:55.770 INFO:teuthology.task.internal:Creating test directory... 2024-04-28T12:26:55.770 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-28T12:26:55.781 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-28T12:26:55.833 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-28T12:26:55.841 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-28T12:26:55.847 INFO:teuthology.task.internal:Creating archive directory... 2024-04-28T12:26:55.847 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-28T12:26:55.852 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-28T12:26:55.959 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-28T12:26:55.967 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-28T12:26:55.968 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-28T12:26:55.975 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-28T12:26:56.060 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T12:26:56.067 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T12:26:56.092 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T12:26:56.096 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T12:26:56.100 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-28T12:26:56.108 INFO:teuthology.task.internal:Configuring sudo... 2024-04-28T12:26:56.108 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-28T12:26:56.113 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-28T12:26:56.191 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-28T12:26:56.201 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-28T12:26:56.201 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-28T12:26:56.206 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-28T12:26:56.262 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T12:26:56.340 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T12:26:56.442 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T12:26:56.518 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T12:26:56.603 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:26:56.603 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-28T12:26:56.688 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T12:26:56.770 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T12:26:56.877 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T12:26:56.958 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T12:26:57.044 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:26:57.045 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-28T12:26:57.125 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-04-28T12:26:57.133 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-04-28T12:26:57.244 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T12:26:57.267 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T12:26:57.555 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-28T12:26:57.566 INFO:teuthology.task.internal:Starting timer... 2024-04-28T12:26:57.566 INFO:teuthology.run_tasks:Running task pcp... 2024-04-28T12:26:57.577 INFO:teuthology.run_tasks:Running task selinux... 2024-04-28T12:26:57.587 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-28T12:26:57.587 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-04-28T12:26:57.707 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-04-28T12:26:57.711 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-04-28T12:26:57.818 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-04-28T12:26:57.823 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-28T12:26:57.824 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-04-28T12:26:57.884 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-04-28T12:26:57.884 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-04-28T12:26:57.943 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-04-28T12:26:57.943 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi131.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-04-28T12:26:57.944 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T12:26:58.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:26:58.014 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T12:26:58.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:26:58.087 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-28T12:26:58.087 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-04-28T12:26:58.155 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-04-28T12:26:58.225 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-28T12:26:58.235 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-28T12:26:58.255 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-28T12:26:58.256 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi131.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-28T12:32:08.753 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-04-28T12:32:08.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-04-28T12:32:08.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:32:08.861 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-04-28T12:32:09.012 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-04-28T12:32:09.012 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-28T12:32:09.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:32:09.118 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-04-28T12:32:09.279 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-28T12:32:09.280 INFO:teuthology.run_tasks:Running task clock... 2024-04-28T12:32:09.290 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-28T12:32:09.291 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-28T12:32:09.291 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T12:32:09.299 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-28T12:32:09.299 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T12:32:09.384 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-28T12:32:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-28T12:32:09.425 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-28T12:32:09.425 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-28T12:32:09.498 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-04-28T12:32:09.498 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-04-28T12:32:09.530 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-04-28T12:32:09.532 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-04-28T12:32:09.574 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-28T12:32:09.575 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-28T12:32:09.614 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-28T12:32:09.616 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-28T12:32:09.754 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-04-28T12:32:09.758 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-04-28T12:32:09.762 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T12:32:09.762 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-04-28T12:32:09.762 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:32:09.762 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:32:09.762 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:32:09.762 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:32:09.766 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T12:32:09.766 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-04-28T12:32:09.766 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:32:09.767 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:32:09.767 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:32:09.767 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:32:09.768 INFO:teuthology.run_tasks:Running task install... 2024-04-28T12:32:09.777 DEBUG:teuthology.task.install:project ceph 2024-04-28T12:32:09.778 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-28T12:32:09.778 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-28T12:32:09.778 INFO:teuthology.task.install:Using flavor: default 2024-04-28T12:32:09.784 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-28T12:32:09.784 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-28T12:32:09.784 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-28T12:32:09.785 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:32:09.787 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-28T12:32:09.787 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:32:09.926 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-28T12:32:09.926 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-28T12:32:09.937 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-28T12:32:09.937 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-28T12:32:10.273 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-28T12:32:10.273 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:32:10.273 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-28T12:32:10.330 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-28T12:32:10.330 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:32:10.330 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-28T12:32:10.362 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-28T12:32:10.363 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-28T12:32:10.416 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-28T12:32:10.416 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-28T12:32:10.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-28T12:32:10.542 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-28T12:32:10.599 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-28T12:32:10.647 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-28T12:32:10.677 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-04-28T12:32:10.681 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-04-28T12:32:10.720 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-04-28T12:32:10.724 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-04-28T12:32:11.006 INFO:teuthology.orchestra.run.smithi157.stdout:48 files removed 2024-04-28T12:32:11.040 INFO:teuthology.orchestra.run.smithi131.stdout:48 files removed 2024-04-28T12:32:11.044 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-28T12:32:11.080 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-28T12:32:12.596 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 75 kB/s | 84 kB 00:01 2024-04-28T12:32:12.630 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 76 kB/s | 84 kB 00:01 2024-04-28T12:32:13.605 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 13 kB/s | 12 kB 00:00 2024-04-28T12:32:13.638 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 12 kB/s | 12 kB 00:00 2024-04-28T12:32:14.523 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 2.3 kB/s | 1.9 kB 00:00 2024-04-28T12:32:14.705 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 1.9 kB/s | 1.9 kB 00:00 2024-04-28T12:32:15.130 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-28T12:32:16.487 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-04-28T12:32:18.900 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-28T12:32:23.380 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-28T12:32:25.122 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-28T12:32:26.591 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-28T12:32:32.482 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-28T12:32:34.850 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T12:32:34.851 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T12:32:34.867 INFO:teuthology.orchestra.run.smithi157.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-28T12:32:34.868 INFO:teuthology.orchestra.run.smithi157.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-28T12:32:34.869 INFO:teuthology.orchestra.run.smithi157.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-28T12:32:34.870 INFO:teuthology.orchestra.run.smithi157.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-28T12:32:34.873 INFO:teuthology.orchestra.run.smithi157.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-28T12:32:34.875 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-28T12:32:34.876 INFO:teuthology.orchestra.run.smithi157.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-28T12:32:34.943 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-28T12:32:34.962 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-04-28T12:32:34.962 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-28T12:32:34.963 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-28T12:32:34.964 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-28T12:32:34.965 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-04-28T12:32:34.966 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-28T12:32:34.966 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T12:32:34.966 INFO:teuthology.orchestra.run.smithi157.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-28T12:32:34.966 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-28T12:32:34.966 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-28T12:32:34.966 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-28T12:32:34.966 INFO:teuthology.orchestra.run.smithi157.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-28T12:32:34.966 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-28T12:32:34.966 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-28T12:32:34.966 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-28T12:32:34.966 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-28T12:32:34.966 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-28T12:32:34.967 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-28T12:32:34.968 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-28T12:32:34.969 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-28T12:32:34.970 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-28T12:32:34.971 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-28T12:32:34.972 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-28T12:32:34.973 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-28T12:32:34.974 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-28T12:32:34.974 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2024-04-28T12:32:34.974 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-28T12:32:34.974 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:32:34.974 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-28T12:32:34.974 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2024-04-28T12:32:34.974 INFO:teuthology.orchestra.run.smithi157.stdout:Install 148 Packages 2024-04-28T12:32:34.974 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 4 Packages 2024-04-28T12:32:34.974 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:32:34.975 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 222 M 2024-04-28T12:32:34.975 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-28T12:32:36.475 INFO:teuthology.orchestra.run.smithi157.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 9.7 kB/s | 6.6 kB 00:00 2024-04-28T12:32:38.129 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 894 kB/s | 19 MB 00:21 2024-04-28T12:32:39.300 INFO:teuthology.orchestra.run.smithi157.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 315 kB/s | 889 kB 00:02 2024-04-28T12:32:39.725 INFO:teuthology.orchestra.run.smithi157.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 342 kB/s | 145 kB 00:00 2024-04-28T12:32:43.118 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.2 MB 00:01 2024-04-28T12:32:45.007 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-28T12:32:46.737 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-28T12:32:49.522 INFO:teuthology.orchestra.run.smithi157.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 247 kB/s | 2.4 MB 00:09 2024-04-28T12:32:52.790 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-04-28T12:32:53.410 INFO:teuthology.orchestra.run.smithi157.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 399 kB/s | 1.5 MB 00:03 2024-04-28T12:32:55.194 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T12:32:55.196 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T12:32:55.215 INFO:teuthology.orchestra.run.smithi131.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-28T12:32:55.216 INFO:teuthology.orchestra.run.smithi131.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-28T12:32:55.217 INFO:teuthology.orchestra.run.smithi131.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-28T12:32:55.218 INFO:teuthology.orchestra.run.smithi131.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-28T12:32:55.221 INFO:teuthology.orchestra.run.smithi131.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-28T12:32:55.224 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-28T12:32:55.225 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-28T12:32:55.295 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-04-28T12:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-04-28T12:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-04-28T12:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-04-28T12:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-04-28T12:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-28T12:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-28T12:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T12:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-28T12:32:55.322 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-28T12:32:55.323 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-28T12:32:55.324 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-28T12:32:55.325 INFO:teuthology.orchestra.run.smithi131.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-28T12:32:55.326 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-28T12:32:55.327 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-28T12:32:55.328 INFO:teuthology.orchestra.run.smithi131.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-28T12:32:55.329 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-28T12:32:55.329 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-28T12:32:55.329 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-28T12:32:55.329 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-28T12:32:55.329 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-28T12:32:55.329 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-28T12:32:55.329 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-28T12:32:55.329 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-28T12:32:55.329 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-28T12:32:55.329 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-28T12:32:55.329 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-28T12:32:55.329 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-28T12:32:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-28T12:32:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-28T12:32:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-28T12:32:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-28T12:32:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-28T12:32:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-28T12:32:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-28T12:32:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-28T12:32:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-28T12:32:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-28T12:32:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-28T12:32:55.330 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-28T12:32:55.331 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-28T12:32:55.332 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-04-28T12:32:55.333 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-28T12:32:55.334 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:32:55.334 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-04-28T12:32:55.334 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-04-28T12:32:55.334 INFO:teuthology.orchestra.run.smithi131.stdout:Install 148 Packages 2024-04-28T12:32:55.334 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 4 Packages 2024-04-28T12:32:55.334 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:32:55.334 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 222 M 2024-04-28T12:32:55.334 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-04-28T12:32:56.290 INFO:teuthology.orchestra.run.smithi157.stdout:(6/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 268 kB/s | 5.4 MB 00:20 2024-04-28T12:32:56.666 INFO:teuthology.orchestra.run.smithi131.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-28T12:32:57.633 INFO:teuthology.orchestra.run.smithi131.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 921 kB/s | 889 kB 00:00 2024-04-28T12:32:57.841 INFO:teuthology.orchestra.run.smithi131.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 700 kB/s | 145 kB 00:00 2024-04-28T12:33:00.917 INFO:teuthology.orchestra.run.smithi131.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 786 kB/s | 2.4 MB 00:03 2024-04-28T12:33:04.016 INFO:teuthology.orchestra.run.smithi131.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 500 kB/s | 1.5 MB 00:03 2024-04-28T12:33:07.208 INFO:teuthology.orchestra.run.smithi131.stdout:(6/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 497 kB/s | 5.4 MB 00:11 2024-04-28T12:33:10.531 INFO:teuthology.orchestra.run.smithi157.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 281 kB/s | 4.7 MB 00:17 2024-04-28T12:33:20.816 INFO:teuthology.orchestra.run.smithi131.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 286 kB/s | 4.7 MB 00:16 2024-04-28T12:33:36.128 INFO:teuthology.orchestra.run.smithi157.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 424 kB/s | 11 MB 00:25 2024-04-28T12:33:36.311 INFO:teuthology.orchestra.run.smithi157.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 138 kB/s | 25 kB 00:00 2024-04-28T12:33:47.843 INFO:teuthology.orchestra.run.smithi157.stdout:(10/152): ceph-common-19.0.0-3306.gbc612ffb.el9 300 kB/s | 21 MB 01:12 2024-04-28T12:33:48.926 INFO:teuthology.orchestra.run.smithi157.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 208 kB/s | 225 kB 00:01 2024-04-28T12:33:49.077 INFO:teuthology.orchestra.run.smithi157.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 181 kB/s | 27 kB 00:00 2024-04-28T12:33:49.352 INFO:teuthology.orchestra.run.smithi157.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 116 kB/s | 32 kB 00:00 2024-04-28T12:33:53.585 INFO:teuthology.orchestra.run.smithi157.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 176 kB/s | 747 kB 00:04 2024-04-28T12:33:54.768 INFO:teuthology.orchestra.run.smithi157.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 138 kB/s | 163 kB 00:01 2024-04-28T12:33:54.895 INFO:teuthology.orchestra.run.smithi131.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 368 kB/s | 21 MB 00:58 2024-04-28T12:33:55.079 INFO:teuthology.orchestra.run.smithi131.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 138 kB/s | 25 kB 00:00 2024-04-28T12:33:55.518 INFO:teuthology.orchestra.run.smithi157.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 169 kB/s | 127 kB 00:00 2024-04-28T12:33:58.168 INFO:teuthology.orchestra.run.smithi157.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 189 kB/s | 501 kB 00:02 2024-04-28T12:34:02.134 INFO:teuthology.orchestra.run.smithi157.stdout:(18/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 262 kB/s | 17 MB 01:05 2024-04-28T12:34:02.418 INFO:teuthology.orchestra.run.smithi157.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 159 kB/s | 45 kB 00:00 2024-04-28T12:34:02.885 INFO:teuthology.orchestra.run.smithi157.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 290 kB/s | 135 kB 00:00 2024-04-28T12:34:03.310 INFO:teuthology.orchestra.run.smithi157.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 381 kB/s | 161 kB 00:00 2024-04-28T12:34:04.352 INFO:teuthology.orchestra.run.smithi157.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 308 kB/s | 321 kB 00:01 2024-04-28T12:34:05.369 INFO:teuthology.orchestra.run.smithi157.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 293 kB/s | 298 kB 00:01 2024-04-28T12:34:05.544 INFO:teuthology.orchestra.run.smithi131.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 242 kB/s | 11 MB 00:44 2024-04-28T12:34:05.719 INFO:teuthology.orchestra.run.smithi157.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 284 kB/s | 99 kB 00:00 2024-04-28T12:34:06.103 INFO:teuthology.orchestra.run.smithi157.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 223 kB/s | 85 kB 00:00 2024-04-28T12:34:07.095 INFO:teuthology.orchestra.run.smithi131.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 145 kB/s | 225 kB 00:01 2024-04-28T12:34:07.403 INFO:teuthology.orchestra.run.smithi131.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 88 kB/s | 27 kB 00:00 2024-04-28T12:34:07.682 INFO:teuthology.orchestra.run.smithi131.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 114 kB/s | 32 kB 00:00 2024-04-28T12:34:11.636 INFO:teuthology.orchestra.run.smithi131.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 189 kB/s | 747 kB 00:03 2024-04-28T12:34:12.669 INFO:teuthology.orchestra.run.smithi131.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 158 kB/s | 163 kB 00:01 2024-04-28T12:34:13.269 INFO:teuthology.orchestra.run.smithi131.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 212 kB/s | 127 kB 00:00 2024-04-28T12:34:15.087 INFO:teuthology.orchestra.run.smithi131.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 276 kB/s | 501 kB 00:01 2024-04-28T12:34:18.468 INFO:teuthology.orchestra.run.smithi157.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 256 kB/s | 3.1 MB 00:12 2024-04-28T12:34:18.901 INFO:teuthology.orchestra.run.smithi157.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 395 kB/s | 171 kB 00:00 2024-04-28T12:34:19.026 INFO:teuthology.orchestra.run.smithi157.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 275 kB/s | 34 kB 00:00 2024-04-28T12:34:19.028 INFO:teuthology.orchestra.run.smithi131.stdout:(18/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 240 kB/s | 17 MB 01:11 2024-04-28T12:34:19.270 INFO:teuthology.orchestra.run.smithi131.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 187 kB/s | 45 kB 00:00 2024-04-28T12:34:19.452 INFO:teuthology.orchestra.run.smithi157.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 335 kB/s | 142 kB 00:00 2024-04-28T12:34:19.720 INFO:teuthology.orchestra.run.smithi131.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 300 kB/s | 135 kB 00:00 2024-04-28T12:34:20.145 INFO:teuthology.orchestra.run.smithi131.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 381 kB/s | 161 kB 00:00 2024-04-28T12:34:20.903 INFO:teuthology.orchestra.run.smithi131.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 424 kB/s | 321 kB 00:00 2024-04-28T12:34:21.571 INFO:teuthology.orchestra.run.smithi131.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 446 kB/s | 298 kB 00:00 2024-04-28T12:34:21.821 INFO:teuthology.orchestra.run.smithi131.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 399 kB/s | 99 kB 00:00 2024-04-28T12:34:22.096 INFO:teuthology.orchestra.run.smithi131.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 310 kB/s | 85 kB 00:00 2024-04-28T12:34:22.510 INFO:teuthology.orchestra.run.smithi157.stdout:(30/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 226 kB/s | 5.4 MB 00:24 2024-04-28T12:34:26.202 INFO:teuthology.orchestra.run.smithi157.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 537 kB/s | 3.5 MB 00:06 2024-04-28T12:34:26.338 INFO:teuthology.orchestra.run.smithi131.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 746 kB/s | 3.1 MB 00:04 2024-04-28T12:34:26.446 INFO:teuthology.orchestra.run.smithi131.stdout:(27/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 484 kB/s | 5.4 MB 00:11 2024-04-28T12:34:26.580 INFO:teuthology.orchestra.run.smithi131.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 258 kB/s | 34 kB 00:00 2024-04-28T12:34:26.688 INFO:teuthology.orchestra.run.smithi131.stdout:(29/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 488 kB/s | 171 kB 00:00 2024-04-28T12:34:26.760 INFO:teuthology.orchestra.run.smithi157.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 446 kB/s | 249 kB 00:00 2024-04-28T12:34:26.939 INFO:teuthology.orchestra.run.smithi131.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 397 kB/s | 142 kB 00:00 2024-04-28T12:34:27.427 INFO:teuthology.orchestra.run.smithi157.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 74 kB/s | 49 kB 00:00 2024-04-28T12:34:27.552 INFO:teuthology.orchestra.run.smithi157.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 132 kB/s | 16 kB 00:00 2024-04-28T12:34:28.511 INFO:teuthology.orchestra.run.smithi157.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 285 kB/s | 273 kB 00:00 2024-04-28T12:34:31.044 INFO:teuthology.orchestra.run.smithi157.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 301 kB/s | 762 kB 00:02 2024-04-28T12:34:31.186 INFO:teuthology.orchestra.run.smithi157.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 129 kB/s | 18 kB 00:00 2024-04-28T12:34:32.219 INFO:teuthology.orchestra.run.smithi157.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 78 kB/s | 80 kB 00:01 2024-04-28T12:34:32.647 INFO:teuthology.orchestra.run.smithi131.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 608 kB/s | 3.5 MB 00:05 2024-04-28T12:34:33.397 INFO:teuthology.orchestra.run.smithi131.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 332 kB/s | 249 kB 00:00 2024-04-28T12:34:33.664 INFO:teuthology.orchestra.run.smithi131.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 185 kB/s | 49 kB 00:00 2024-04-28T12:34:33.823 INFO:teuthology.orchestra.run.smithi131.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 104 kB/s | 16 kB 00:00 2024-04-28T12:34:34.453 INFO:teuthology.orchestra.run.smithi157.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 360 kB/s | 803 kB 00:02 2024-04-28T12:34:34.540 INFO:teuthology.orchestra.run.smithi131.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 382 kB/s | 273 kB 00:00 2024-04-28T12:34:35.567 INFO:teuthology.orchestra.run.smithi157.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 176 kB/s | 195 kB 00:01 2024-04-28T12:34:36.690 INFO:teuthology.orchestra.run.smithi131.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 355 kB/s | 762 kB 00:02 2024-04-28T12:34:36.823 INFO:teuthology.orchestra.run.smithi131.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 138 kB/s | 18 kB 00:00 2024-04-28T12:34:36.844 INFO:teuthology.orchestra.run.smithi157.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 198 kB/s | 253 kB 00:01 2024-04-28T12:34:37.407 INFO:teuthology.orchestra.run.smithi131.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 138 kB/s | 80 kB 00:00 2024-04-28T12:34:38.047 INFO:teuthology.orchestra.run.smithi131.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-04-28T12:34:38.191 INFO:teuthology.orchestra.run.smithi131.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-28T12:34:38.341 INFO:teuthology.orchestra.run.smithi131.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-28T12:34:38.700 INFO:teuthology.orchestra.run.smithi131.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 3.4 MB/s | 1.2 MB 00:00 2024-04-28T12:34:38.834 INFO:teuthology.orchestra.run.smithi131.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 804 kB/s | 106 kB 00:00 2024-04-28T12:34:38.967 INFO:teuthology.orchestra.run.smithi131.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-04-28T12:34:39.101 INFO:teuthology.orchestra.run.smithi131.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 941 kB/s | 125 kB 00:00 2024-04-28T12:34:39.251 INFO:teuthology.orchestra.run.smithi131.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.4 MB/s | 215 kB 00:00 2024-04-28T12:34:39.476 INFO:teuthology.orchestra.run.smithi131.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-04-28T12:34:39.568 INFO:teuthology.orchestra.run.smithi131.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-28T12:34:39.621 INFO:teuthology.orchestra.run.smithi131.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.0 MB/s | 107 kB 00:00 2024-04-28T12:34:39.671 INFO:teuthology.orchestra.run.smithi131.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 435 kB/s | 21 kB 00:00 2024-04-28T12:34:39.721 INFO:teuthology.orchestra.run.smithi131.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 6.2 MB/s | 313 kB 00:00 2024-04-28T12:34:39.772 INFO:teuthology.orchestra.run.smithi131.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 662 kB/s | 33 kB 00:00 2024-04-28T12:34:39.837 INFO:teuthology.orchestra.run.smithi157.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 420 kB/s | 1.2 MB 00:02 2024-04-28T12:34:39.905 INFO:teuthology.orchestra.run.smithi131.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 23 MB/s | 3.0 MB 00:00 2024-04-28T12:34:39.947 INFO:teuthology.orchestra.run.smithi131.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 442 kB/s | 18 kB 00:00 2024-04-28T12:34:40.012 INFO:teuthology.orchestra.run.smithi131.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 175 kB/s | 11 kB 00:00 2024-04-28T12:34:40.065 INFO:teuthology.orchestra.run.smithi131.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.0 MB/s | 160 kB 00:00 2024-04-28T12:34:40.115 INFO:teuthology.orchestra.run.smithi131.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 916 kB/s | 45 kB 00:00 2024-04-28T12:34:40.128 INFO:teuthology.orchestra.run.smithi157.stdout:(43/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 788 kB/s | 49 MB 01:03 2024-04-28T12:34:40.182 INFO:teuthology.orchestra.run.smithi131.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 662 kB 00:00 2024-04-28T12:34:40.249 INFO:teuthology.orchestra.run.smithi131.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 582 kB 00:00 2024-04-28T12:34:40.299 INFO:teuthology.orchestra.run.smithi131.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 516 kB/s | 25 kB 00:00 2024-04-28T12:34:40.350 INFO:teuthology.orchestra.run.smithi131.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 962 kB/s | 47 kB 00:00 2024-04-28T12:34:40.400 INFO:teuthology.orchestra.run.smithi131.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.8 MB/s | 292 kB 00:00 2024-04-28T12:34:40.450 INFO:teuthology.orchestra.run.smithi131.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-28T12:34:40.517 INFO:teuthology.orchestra.run.smithi131.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 534 kB/s | 35 kB 00:00 2024-04-28T12:34:40.684 INFO:teuthology.orchestra.run.smithi131.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 29 MB/s | 4.7 MB 00:00 2024-04-28T12:34:40.843 INFO:teuthology.orchestra.run.smithi131.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-04-28T12:34:40.935 INFO:teuthology.orchestra.run.smithi131.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 18 MB/s | 1.6 MB 00:00 2024-04-28T12:34:41.010 INFO:teuthology.orchestra.run.smithi131.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-28T12:34:41.061 INFO:teuthology.orchestra.run.smithi131.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-04-28T12:34:41.153 INFO:teuthology.orchestra.run.smithi157.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 81 kB/s | 106 kB 00:01 2024-04-28T12:34:41.236 INFO:teuthology.orchestra.run.smithi131.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-04-28T12:34:41.270 INFO:teuthology.orchestra.run.smithi157.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 118 kB/s | 135 kB 00:01 2024-04-28T12:34:41.311 INFO:teuthology.orchestra.run.smithi131.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-04-28T12:34:41.362 INFO:teuthology.orchestra.run.smithi131.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-28T12:34:41.545 INFO:teuthology.orchestra.run.smithi131.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 6.0 MB 00:00 2024-04-28T12:34:41.596 INFO:teuthology.orchestra.run.smithi131.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-28T12:34:41.646 INFO:teuthology.orchestra.run.smithi131.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-28T12:34:41.696 INFO:teuthology.orchestra.run.smithi131.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 517 kB/s | 26 kB 00:00 2024-04-28T12:34:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 388 kB/s | 19 kB 00:00 2024-04-28T12:34:41.797 INFO:teuthology.orchestra.run.smithi131.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-28T12:34:41.847 INFO:teuthology.orchestra.run.smithi131.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 704 kB/s | 35 kB 00:00 2024-04-28T12:34:42.023 INFO:teuthology.orchestra.run.smithi131.stdout:(80/152): ceph-mgr-diskprediction-local-19.0.0- 503 kB/s | 7.4 MB 00:15 2024-04-28T12:34:42.173 INFO:teuthology.orchestra.run.smithi131.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-28T12:34:42.195 INFO:teuthology.orchestra.run.smithi157.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 120 kB/s | 125 kB 00:01 2024-04-28T12:34:42.223 INFO:teuthology.orchestra.run.smithi131.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-28T12:34:42.249 INFO:teuthology.orchestra.run.smithi131.stdout:(83/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-04-28T12:34:42.274 INFO:teuthology.orchestra.run.smithi131.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-28T12:34:42.299 INFO:teuthology.orchestra.run.smithi131.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 856 kB/s | 42 kB 00:00 2024-04-28T12:34:42.333 INFO:teuthology.orchestra.run.smithi131.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-04-28T12:34:42.366 INFO:teuthology.orchestra.run.smithi131.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-28T12:34:42.408 INFO:teuthology.orchestra.run.smithi131.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-28T12:34:42.421 INFO:teuthology.orchestra.run.smithi157.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-04-28T12:34:42.434 INFO:teuthology.orchestra.run.smithi131.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 768 kB/s | 51 kB 00:00 2024-04-28T12:34:42.459 INFO:teuthology.orchestra.run.smithi131.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-28T12:34:42.509 INFO:teuthology.orchestra.run.smithi131.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 847 kB/s | 42 kB 00:00 2024-04-28T12:34:42.513 INFO:teuthology.orchestra.run.smithi157.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-28T12:34:42.546 INFO:teuthology.orchestra.run.smithi157.stdout:(49/152): python3-urllib3-1.26.5-5.el9.noarch.r 168 kB/s | 215 kB 00:01 2024-04-28T12:34:42.551 INFO:teuthology.orchestra.run.smithi131.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-04-28T12:34:42.571 INFO:teuthology.orchestra.run.smithi157.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-28T12:34:42.610 INFO:teuthology.orchestra.run.smithi131.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-28T12:34:42.630 INFO:teuthology.orchestra.run.smithi157.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-04-28T12:34:42.652 INFO:teuthology.orchestra.run.smithi131.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 874 kB/s | 36 kB 00:00 2024-04-28T12:34:42.655 INFO:teuthology.orchestra.run.smithi157.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-04-28T12:34:42.681 INFO:teuthology.orchestra.run.smithi157.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 663 kB/s | 33 kB 00:00 2024-04-28T12:34:42.694 INFO:teuthology.orchestra.run.smithi131.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-28T12:34:42.731 INFO:teuthology.orchestra.run.smithi157.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 369 kB/s | 18 kB 00:00 2024-04-28T12:34:42.753 INFO:teuthology.orchestra.run.smithi131.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-28T12:34:42.795 INFO:teuthology.orchestra.run.smithi131.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-04-28T12:34:42.801 INFO:teuthology.orchestra.run.smithi157.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 161 kB/s | 11 kB 00:00 2024-04-28T12:34:42.846 INFO:teuthology.orchestra.run.smithi131.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-28T12:34:42.856 INFO:teuthology.orchestra.run.smithi157.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.9 MB/s | 160 kB 00:00 2024-04-28T12:34:42.904 INFO:teuthology.orchestra.run.smithi131.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-28T12:34:42.949 INFO:teuthology.orchestra.run.smithi157.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-04-28T12:34:42.955 INFO:teuthology.orchestra.run.smithi131.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 911 kB/s | 45 kB 00:00 2024-04-28T12:34:42.973 INFO:teuthology.orchestra.run.smithi157.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 389 kB/s | 45 kB 00:00 2024-04-28T12:34:43.016 INFO:teuthology.orchestra.run.smithi157.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 662 kB 00:00 2024-04-28T12:34:43.066 INFO:teuthology.orchestra.run.smithi157.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-04-28T12:34:43.091 INFO:teuthology.orchestra.run.smithi157.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 342 kB/s | 25 kB 00:00 2024-04-28T12:34:43.117 INFO:teuthology.orchestra.run.smithi157.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 961 kB/s | 47 kB 00:00 2024-04-28T12:34:43.150 INFO:teuthology.orchestra.run.smithi157.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-28T12:34:43.184 INFO:teuthology.orchestra.run.smithi157.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-28T12:34:43.188 INFO:teuthology.orchestra.run.smithi131.stdout:(101/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-28T12:34:43.209 INFO:teuthology.orchestra.run.smithi157.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 35 kB 00:00 2024-04-28T12:34:43.214 INFO:teuthology.orchestra.run.smithi131.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-04-28T12:34:43.360 INFO:teuthology.orchestra.run.smithi157.stdout:(66/152): ceph-mgr-diskprediction-local-19.0.0- 364 kB/s | 7.4 MB 00:20 2024-04-28T12:34:43.493 INFO:teuthology.orchestra.run.smithi157.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-28T12:34:43.602 INFO:teuthology.orchestra.run.smithi157.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-28T12:34:43.660 INFO:teuthology.orchestra.run.smithi157.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.4 MB/s | 911 kB 00:00 2024-04-28T12:34:43.694 INFO:teuthology.orchestra.run.smithi157.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-28T12:34:43.769 INFO:teuthology.orchestra.run.smithi157.stdout:(71/152): postgresql-13.14-1.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-04-28T12:34:43.853 INFO:teuthology.orchestra.run.smithi157.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-04-28T12:34:43.878 INFO:teuthology.orchestra.run.smithi157.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 810 kB/s | 87 kB 00:00 2024-04-28T12:34:43.937 INFO:teuthology.orchestra.run.smithi157.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-28T12:34:43.939 INFO:teuthology.orchestra.run.smithi131.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 757 kB/s | 548 kB 00:00 2024-04-28T12:34:43.995 INFO:teuthology.orchestra.run.smithi157.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-28T12:34:44.106 INFO:teuthology.orchestra.run.smithi131.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.8 MB/s | 309 kB 00:00 2024-04-28T12:34:44.112 INFO:teuthology.orchestra.run.smithi157.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-28T12:34:44.137 INFO:teuthology.orchestra.run.smithi157.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 182 kB/s | 26 kB 00:00 2024-04-28T12:34:44.163 INFO:teuthology.orchestra.run.smithi157.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 388 kB/s | 19 kB 00:00 2024-04-28T12:34:44.196 INFO:teuthology.orchestra.run.smithi157.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-28T12:34:44.222 INFO:teuthology.orchestra.run.smithi157.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 602 kB/s | 35 kB 00:00 2024-04-28T12:34:44.231 INFO:teuthology.orchestra.run.smithi131.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 156 kB/s | 19 kB 00:00 2024-04-28T12:34:44.280 INFO:teuthology.orchestra.run.smithi157.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-04-28T12:34:44.282 INFO:teuthology.orchestra.run.smithi131.stdout:(106/152): protobuf-compiler-3.14.0-13.el9.x86_ 791 kB/s | 863 kB 00:01 2024-04-28T12:34:44.397 INFO:teuthology.orchestra.run.smithi157.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-28T12:34:44.431 INFO:teuthology.orchestra.run.smithi157.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-04-28T12:34:44.456 INFO:teuthology.orchestra.run.smithi157.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-28T12:34:44.481 INFO:teuthology.orchestra.run.smithi157.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 855 kB/s | 42 kB 00:00 2024-04-28T12:34:44.515 INFO:teuthology.orchestra.run.smithi157.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-04-28T12:34:44.549 INFO:teuthology.orchestra.run.smithi157.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-28T12:34:44.582 INFO:teuthology.orchestra.run.smithi157.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-28T12:34:44.607 INFO:teuthology.orchestra.run.smithi157.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 883 kB/s | 51 kB 00:00 2024-04-28T12:34:44.615 INFO:teuthology.orchestra.run.smithi131.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 78 kB/s | 26 kB 00:00 2024-04-28T12:34:44.633 INFO:teuthology.orchestra.run.smithi157.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-28T12:34:44.683 INFO:teuthology.orchestra.run.smithi157.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 847 kB/s | 42 kB 00:00 2024-04-28T12:34:44.765 INFO:teuthology.orchestra.run.smithi131.stdout:(108/152): libarrow-9.0.0-11.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2024-04-28T12:34:44.800 INFO:teuthology.orchestra.run.smithi157.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.5 MB/s | 5.1 MB 00:00 2024-04-28T12:34:44.847 INFO:teuthology.orchestra.run.smithi157.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 271 kB/s | 44 kB 00:00 2024-04-28T12:34:44.865 INFO:teuthology.orchestra.run.smithi131.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 195 kB/s | 49 kB 00:00 2024-04-28T12:34:44.884 INFO:teuthology.orchestra.run.smithi157.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-28T12:34:44.891 INFO:teuthology.orchestra.run.smithi131.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 540 kB/s | 67 kB 00:00 2024-04-28T12:34:44.909 INFO:teuthology.orchestra.run.smithi157.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 587 kB/s | 36 kB 00:00 2024-04-28T12:34:44.934 INFO:teuthology.orchestra.run.smithi157.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 960 kB/s | 47 kB 00:00 2024-04-28T12:34:44.968 INFO:teuthology.orchestra.run.smithi157.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-28T12:34:44.993 INFO:teuthology.orchestra.run.smithi157.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-28T12:34:45.019 INFO:teuthology.orchestra.run.smithi157.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-28T12:34:45.041 INFO:teuthology.orchestra.run.smithi131.stdout:(111/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.5 MB/s | 839 kB 00:00 2024-04-28T12:34:45.044 INFO:teuthology.orchestra.run.smithi157.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-28T12:34:45.069 INFO:teuthology.orchestra.run.smithi157.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-28T12:34:45.200 INFO:teuthology.orchestra.run.smithi131.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-04-28T12:34:45.225 INFO:teuthology.orchestra.run.smithi131.stdout:(113/152): luarocks-3.9.2-1.el9.noarch.rpm 421 kB/s | 151 kB 00:00 2024-04-28T12:34:45.317 INFO:teuthology.orchestra.run.smithi131.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 374 kB/s | 43 kB 00:00 2024-04-28T12:34:45.342 INFO:teuthology.orchestra.run.smithi131.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2024-04-28T12:34:45.378 INFO:teuthology.orchestra.run.smithi157.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-28T12:34:45.395 INFO:teuthology.orchestra.run.smithi157.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 64 kB/s | 22 kB 00:00 2024-04-28T12:34:45.451 INFO:teuthology.orchestra.run.smithi131.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 113 kB/s | 15 kB 00:00 2024-04-28T12:34:45.535 INFO:teuthology.orchestra.run.smithi131.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 897 kB/s | 172 kB 00:00 2024-04-28T12:34:45.577 INFO:teuthology.orchestra.run.smithi131.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-28T12:34:45.710 INFO:teuthology.orchestra.run.smithi131.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 266 kB/s | 35 kB 00:00 2024-04-28T12:34:45.752 INFO:teuthology.orchestra.run.smithi131.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.4 MB/s | 303 kB 00:00 2024-04-28T12:34:45.853 INFO:teuthology.orchestra.run.smithi131.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.4 MB/s | 206 kB 00:00 2024-04-28T12:34:45.978 INFO:teuthology.orchestra.run.smithi157.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 915 kB/s | 548 kB 00:00 2024-04-28T12:34:45.986 INFO:teuthology.orchestra.run.smithi131.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-28T12:34:46.012 INFO:teuthology.orchestra.run.smithi157.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 501 kB/s | 309 kB 00:00 2024-04-28T12:34:46.071 INFO:teuthology.orchestra.run.smithi157.stdout:(106/152): protobuf-compiler-3.14.0-13.el9.x86_ 863 kB/s | 863 kB 00:01 2024-04-28T12:34:46.096 INFO:teuthology.orchestra.run.smithi157.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 168 kB/s | 19 kB 00:00 2024-04-28T12:34:46.103 INFO:teuthology.orchestra.run.smithi131.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-28T12:34:46.213 INFO:teuthology.orchestra.run.smithi131.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.4 MB/s | 2.0 MB 00:00 2024-04-28T12:34:46.238 INFO:teuthology.orchestra.run.smithi131.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 133 kB/s | 18 kB 00:00 2024-04-28T12:34:46.346 INFO:teuthology.orchestra.run.smithi131.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 174 kB/s | 23 kB 00:00 2024-04-28T12:34:46.374 INFO:teuthology.orchestra.run.smithi131.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 146 kB/s | 19 kB 00:00 2024-04-28T12:34:46.472 INFO:teuthology.orchestra.run.smithi131.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 158 kB/s | 20 kB 00:00 2024-04-28T12:34:46.530 INFO:teuthology.orchestra.run.smithi131.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2024-04-28T12:34:46.597 INFO:teuthology.orchestra.run.smithi131.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 372 kB/s | 46 kB 00:00 2024-04-28T12:34:46.648 INFO:teuthology.orchestra.run.smithi131.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 677 kB/s | 79 kB 00:00 2024-04-28T12:34:46.740 INFO:teuthology.orchestra.run.smithi131.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 408 kB/s | 58 kB 00:00 2024-04-28T12:34:46.857 INFO:teuthology.orchestra.run.smithi131.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-04-28T12:34:46.924 INFO:teuthology.orchestra.run.smithi131.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 990 kB/s | 272 kB 00:00 2024-04-28T12:34:47.057 INFO:teuthology.orchestra.run.smithi131.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 213 kB/s | 28 kB 00:00 2024-04-28T12:34:47.099 INFO:teuthology.orchestra.run.smithi131.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 372 kB/s | 90 kB 00:00 2024-04-28T12:34:47.166 INFO:teuthology.orchestra.run.smithi131.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 285 kB/s | 31 kB 00:00 2024-04-28T12:34:47.225 INFO:teuthology.orchestra.run.smithi131.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-28T12:34:47.292 INFO:teuthology.orchestra.run.smithi131.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 474 kB/s | 59 kB 00:00 2024-04-28T12:34:47.334 INFO:teuthology.orchestra.run.smithi131.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 331 kB/s | 36 kB 00:00 2024-04-28T12:34:47.401 INFO:teuthology.orchestra.run.smithi131.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 472 kB/s | 51 kB 00:00 2024-04-28T12:34:47.485 INFO:teuthology.orchestra.run.smithi131.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-04-28T12:34:47.535 INFO:teuthology.orchestra.run.smithi131.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 674 kB/s | 90 kB 00:00 2024-04-28T12:34:47.635 INFO:teuthology.orchestra.run.smithi131.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-04-28T12:34:47.654 INFO:teuthology.orchestra.run.smithi157.stdout:(108/152): libarrow-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-04-28T12:34:47.660 INFO:teuthology.orchestra.run.smithi131.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-04-28T12:34:47.672 INFO:teuthology.orchestra.run.smithi157.stdout:(109/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 16 kB/s | 26 kB 00:01 2024-04-28T12:34:47.697 INFO:teuthology.orchestra.run.smithi157.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 30 kB/s | 49 kB 00:01 2024-04-28T12:34:47.755 INFO:teuthology.orchestra.run.smithi157.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 821 kB/s | 67 kB 00:00 2024-04-28T12:34:47.777 INFO:teuthology.orchestra.run.smithi131.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-04-28T12:34:47.889 INFO:teuthology.orchestra.run.smithi157.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 701 kB/s | 151 kB 00:00 2024-04-28T12:34:47.961 INFO:teuthology.orchestra.run.smithi131.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-04-28T12:34:48.339 INFO:teuthology.orchestra.run.smithi157.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.3 MB/s | 839 kB 00:00 2024-04-28T12:34:48.631 INFO:teuthology.orchestra.run.smithi157.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 626 kB/s | 548 kB 00:00 2024-04-28T12:34:48.656 INFO:teuthology.orchestra.run.smithi157.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 57 kB/s | 43 kB 00:00 2024-04-28T12:34:48.681 INFO:teuthology.orchestra.run.smithi157.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 94 kB/s | 32 kB 00:00 2024-04-28T12:34:48.707 INFO:teuthology.orchestra.run.smithi157.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2024-04-28T12:34:48.824 INFO:teuthology.orchestra.run.smithi157.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-04-28T12:34:49.082 INFO:teuthology.orchestra.run.smithi157.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 897 kB/s | 358 kB 00:00 2024-04-28T12:34:49.241 INFO:teuthology.orchestra.run.smithi157.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 569 kB/s | 303 kB 00:00 2024-04-28T12:34:49.266 INFO:teuthology.orchestra.run.smithi157.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 80 kB/s | 35 kB 00:00 2024-04-28T12:34:49.391 INFO:teuthology.orchestra.run.smithi157.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 668 kB/s | 206 kB 00:00 2024-04-28T12:34:50.170 INFO:teuthology.orchestra.run.smithi131.stdout:(148/152): ceph-test-19.0.0-3306.gbc612ffb.el9. 913 kB/s | 49 MB 00:55 2024-04-28T12:34:51.245 INFO:teuthology.orchestra.run.smithi131.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-04-28T12:34:51.492 INFO:teuthology.orchestra.run.smithi157.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 928 kB/s | 2.0 MB 00:02 2024-04-28T12:34:51.537 INFO:teuthology.orchestra.run.smithi131.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.2 MB/s | 2.4 MB 00:00 2024-04-28T12:34:51.592 INFO:teuthology.orchestra.run.smithi157.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 62 kB/s | 144 kB 00:02 2024-04-28T12:34:51.625 INFO:teuthology.orchestra.run.smithi157.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 4.8 kB/s | 11 kB 00:02 2024-04-28T12:34:51.650 INFO:teuthology.orchestra.run.smithi157.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 112 kB/s | 18 kB 00:00 2024-04-28T12:34:51.676 INFO:teuthology.orchestra.run.smithi157.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-04-28T12:34:51.709 INFO:teuthology.orchestra.run.smithi157.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 234 kB/s | 19 kB 00:00 2024-04-28T12:34:51.735 INFO:teuthology.orchestra.run.smithi157.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 237 kB/s | 20 kB 00:00 2024-04-28T12:34:51.920 INFO:teuthology.orchestra.run.smithi131.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 840 kB/s | 3.4 MB 00:04 2024-04-28T12:34:52.726 INFO:teuthology.orchestra.run.smithi157.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 996 kB/s | 1.0 MB 00:01 2024-04-28T12:34:52.768 INFO:teuthology.orchestra.run.smithi157.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 44 kB/s | 46 kB 00:01 2024-04-28T12:34:52.821 INFO:teuthology.orchestra.run.smithi157.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 73 kB/s | 79 kB 00:01 2024-04-28T12:34:52.896 INFO:teuthology.orchestra.run.smithi157.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 345 kB/s | 58 kB 00:00 2024-04-28T12:34:53.138 INFO:teuthology.orchestra.run.smithi157.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 739 kB/s | 272 kB 00:00 2024-04-28T12:34:53.180 INFO:teuthology.orchestra.run.smithi157.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 46 kB/s | 16 kB 00:00 2024-04-28T12:34:53.264 INFO:teuthology.orchestra.run.smithi157.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 246 kB/s | 90 kB 00:00 2024-04-28T12:34:53.289 INFO:teuthology.orchestra.run.smithi157.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 189 kB/s | 28 kB 00:00 2024-04-28T12:34:53.314 INFO:teuthology.orchestra.run.smithi157.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 231 kB/s | 31 kB 00:00 2024-04-28T12:34:53.439 INFO:teuthology.orchestra.run.smithi157.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-04-28T12:34:53.464 INFO:teuthology.orchestra.run.smithi157.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 338 kB/s | 59 kB 00:00 2024-04-28T12:34:53.498 INFO:teuthology.orchestra.run.smithi157.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 195 kB/s | 36 kB 00:00 2024-04-28T12:34:53.540 INFO:teuthology.orchestra.run.smithi157.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 511 kB/s | 51 kB 00:00 2024-04-28T12:34:53.690 INFO:teuthology.orchestra.run.smithi157.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.0 MB/s | 230 kB 00:00 2024-04-28T12:34:53.765 INFO:teuthology.orchestra.run.smithi157.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 337 kB/s | 90 kB 00:00 2024-04-28T12:34:54.103 INFO:teuthology.orchestra.run.smithi131.stdout:(152/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 524 kB/s | 3.1 MB 00:06 2024-04-28T12:34:54.111 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-04-28T12:34:54.112 INFO:teuthology.orchestra.run.smithi131.stdout:Total 1.9 MB/s | 222 MB 01:58 2024-04-28T12:34:54.116 INFO:teuthology.orchestra.run.smithi157.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 743 kB/s | 427 kB 00:00 2024-04-28T12:34:54.141 INFO:teuthology.orchestra.run.smithi157.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 44 kB/s | 20 kB 00:00 2024-04-28T12:34:54.275 INFO:teuthology.orchestra.run.smithi157.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 377 kB/s | 192 kB 00:00 2024-04-28T12:34:55.458 INFO:teuthology.orchestra.run.smithi157.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 1.2 MB/s | 1.6 MB 00:01 2024-04-28T12:34:56.300 INFO:teuthology.orchestra.run.smithi157.stdout:(149/152): librados2-19.0.0-3306.gbc612ffb.el9. 1.6 MB/s | 3.4 MB 00:02 2024-04-28T12:34:56.970 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-04-28T12:34:57.227 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-04-28T12:34:57.227 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-04-28T12:34:58.450 INFO:teuthology.orchestra.run.smithi157.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 462 kB/s | 1.3 MB 00:02 2024-04-28T12:34:59.954 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-04-28T12:34:59.954 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-04-28T12:35:00.883 INFO:teuthology.orchestra.run.smithi157.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 535 kB/s | 2.4 MB 00:04 2024-04-28T12:35:01.608 INFO:teuthology.orchestra.run.smithi157.stdout:(152/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 439 kB/s | 3.1 MB 00:07 2024-04-28T12:35:01.617 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-28T12:35:01.617 INFO:teuthology.orchestra.run.smithi157.stdout:Total 1.5 MB/s | 222 MB 02:26 2024-04-28T12:35:04.493 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-28T12:35:04.739 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-28T12:35:04.739 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-28T12:35:06.330 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-04-28T12:35:06.435 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-28T12:35:06.507 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-28T12:35:06.630 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-28T12:35:06.717 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-28T12:35:06.754 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-28T12:35:06.887 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-28T12:35:07.316 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-28T12:35:07.418 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:35:07.427 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-28T12:35:07.428 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-28T12:35:07.567 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:35:07.628 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:35:07.755 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:35:07.802 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T12:35:08.136 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-28T12:35:08.208 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-28T12:35:08.232 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:35:08.378 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:35:08.408 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T12:35:08.502 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T12:35:08.535 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T12:35:08.637 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T12:35:08.922 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T12:35:08.977 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T12:35:09.174 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-28T12:35:09.312 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-28T12:35:09.361 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-28T12:35:09.409 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-28T12:35:09.510 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-28T12:35:09.577 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-28T12:35:09.654 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-28T12:35:09.712 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-28T12:35:09.752 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-28T12:35:09.866 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-28T12:35:09.942 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-28T12:35:09.978 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-28T12:35:10.033 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-28T12:35:10.159 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-28T12:35:10.198 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-28T12:35:10.248 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-28T12:35:10.279 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-28T12:35:10.390 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-28T12:35:10.459 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-28T12:35:10.567 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-28T12:35:10.751 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-28T12:35:10.821 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-28T12:35:11.499 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-28T12:35:11.687 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-28T12:35:11.837 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-28T12:35:11.897 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-28T12:35:11.947 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-28T12:35:12.023 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-28T12:35:12.300 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-28T12:35:12.366 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-28T12:35:12.443 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-28T12:35:12.574 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-28T12:35:12.623 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-28T12:35:12.662 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-28T12:35:12.729 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-28T12:35:12.768 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-28T12:35:12.823 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-28T12:35:12.880 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-28T12:35:13.066 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-28T12:35:13.117 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-28T12:35:13.156 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-28T12:35:13.203 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-28T12:35:13.330 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-28T12:35:13.373 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-28T12:35:13.421 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-28T12:35:13.455 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-28T12:35:13.510 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-28T12:35:13.698 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-28T12:35:13.738 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-28T12:35:13.840 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-28T12:35:13.873 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-28T12:35:13.913 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-28T12:35:13.920 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-28T12:35:13.960 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-28T12:35:14.011 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-28T12:35:14.038 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-28T12:35:14.074 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-28T12:35:14.101 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-28T12:35:14.124 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-28T12:35:14.158 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-28T12:35:14.200 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-28T12:35:14.246 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-28T12:35:14.257 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-28T12:35:14.297 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-28T12:35:14.371 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-28T12:35:14.420 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-28T12:35:14.561 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-28T12:35:14.628 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-28T12:35:14.668 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-28T12:35:14.690 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-28T12:35:14.738 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-28T12:35:14.789 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:35:14.885 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-28T12:35:14.948 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-28T12:35:14.948 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:35:14.981 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:35:15.090 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:35:15.160 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T12:35:15.526 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-28T12:35:15.594 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-28T12:35:15.618 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:35:15.769 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:35:15.803 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T12:35:15.889 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T12:35:15.921 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T12:35:15.928 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-28T12:35:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-28T12:35:15.991 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T12:35:16.022 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-28T12:35:16.146 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-28T12:35:16.216 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-28T12:35:16.310 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T12:35:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-28T12:35:16.357 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-28T12:35:16.363 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T12:35:16.559 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-28T12:35:16.589 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-28T12:35:16.621 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-28T12:35:16.682 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-28T12:35:16.721 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-28T12:35:16.730 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-28T12:35:16.769 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-28T12:35:16.870 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-28T12:35:16.924 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-28T12:35:16.986 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-28T12:35:17.038 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-28T12:35:17.078 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-28T12:35:17.194 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-28T12:35:17.267 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-28T12:35:17.305 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-28T12:35:17.363 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-28T12:35:17.468 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-28T12:35:17.516 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-28T12:35:17.532 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-28T12:35:17.572 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-28T12:35:17.606 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-28T12:35:17.676 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-28T12:35:17.725 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-28T12:35:17.787 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-28T12:35:17.893 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-28T12:35:18.076 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-28T12:35:18.145 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-28T12:35:18.802 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-28T12:35:18.918 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-28T12:35:18.962 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-28T12:35:19.009 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-28T12:35:19.057 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-28T12:35:19.131 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-28T12:35:19.244 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-28T12:35:19.346 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-28T12:35:19.369 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-28T12:35:19.416 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-28T12:35:19.423 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-28T12:35:19.469 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-28T12:35:19.496 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-28T12:35:19.590 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-28T12:35:19.639 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-28T12:35:19.678 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-28T12:35:19.747 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-28T12:35:19.787 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-28T12:35:19.848 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-28T12:35:19.862 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-28T12:35:19.908 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-28T12:35:19.909 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T12:35:20.029 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T12:35:20.065 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-28T12:35:20.113 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-28T12:35:20.122 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-28T12:35:20.168 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-28T12:35:20.205 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-28T12:35:20.253 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-28T12:35:20.379 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-28T12:35:20.430 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-28T12:35:20.479 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-28T12:35:20.511 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-28T12:35:20.545 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-28T12:35:20.689 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-28T12:35:20.721 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T12:35:20.744 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-28T12:35:20.804 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T12:35:20.856 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-28T12:35:20.921 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-28T12:35:20.960 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-28T12:35:20.998 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-28T12:35:21.043 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-28T12:35:21.102 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-28T12:35:21.153 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-28T12:35:21.196 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-28T12:35:21.241 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-28T12:35:21.291 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-28T12:35:21.367 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-28T12:35:21.431 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-28T12:35:21.576 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-28T12:35:21.647 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-28T12:35:21.690 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-28T12:35:21.762 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-28T12:35:21.916 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-28T12:35:21.985 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-28T12:35:22.951 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-28T12:35:23.001 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-28T12:35:23.035 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-28T12:35:23.222 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-28T12:35:23.324 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-28T12:35:23.369 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-28T12:35:23.526 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:35:23.570 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:35:23.602 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-28T12:35:23.633 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-28T12:35:23.737 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-28T12:35:24.217 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:35:24.256 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T12:35:24.427 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T12:35:24.427 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-28T12:35:24.427 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-28T12:35:24.428 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:35:24.474 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T12:35:24.561 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-28T12:35:24.704 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-28T12:35:26.323 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-28T12:35:26.423 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-28T12:35:26.464 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-28T12:35:26.507 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-28T12:35:26.883 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-28T12:35:26.927 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T12:35:27.047 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T12:35:27.083 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-28T12:35:27.147 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-28T12:35:27.691 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-28T12:35:27.722 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T12:35:27.813 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T12:35:27.840 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-28T12:35:30.526 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:35:30.569 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:35:31.212 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:35:31.265 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T12:35:31.424 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T12:35:31.424 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-28T12:35:31.425 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-28T12:35:31.425 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:35:31.466 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T12:35:37.516 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T12:35:37.516 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-04-28T12:35:37.516 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-04-28T12:35:37.516 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-04-28T12:35:37.516 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-04-28T12:35:37.516 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-04-28T12:35:37.516 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /root 2024-04-28T12:35:37.517 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-04-28T12:35:37.517 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:35:37.759 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T12:35:37.811 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T12:35:37.811 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:37.812 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-28T12:35:37.812 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:35:39.453 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T12:35:39.534 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T12:35:39.534 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:39.534 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-28T12:35:39.534 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T12:35:39.534 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T12:35:39.534 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:35:39.779 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T12:35:39.852 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T12:35:39.852 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:39.853 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-28T12:35:39.853 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T12:35:39.853 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T12:35:39.853 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:35:40.329 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T12:35:40.398 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T12:35:40.398 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:40.398 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-28T12:35:40.398 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T12:35:40.398 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T12:35:40.398 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:35:40.521 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-28T12:35:40.582 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-28T12:35:40.682 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-28T12:35:40.919 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-28T12:35:40.976 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-28T12:35:41.061 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-28T12:35:41.187 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T12:35:41.353 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T12:35:41.401 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-28T12:35:41.440 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-28T12:35:41.483 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-28T12:35:42.195 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T12:35:42.245 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T12:35:42.303 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-28T12:35:42.410 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-28T12:35:42.455 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T12:35:43.774 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T12:35:43.807 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T12:35:44.560 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T12:35:44.560 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-04-28T12:35:44.560 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-04-28T12:35:44.560 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-04-28T12:35:44.560 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-04-28T12:35:44.560 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-04-28T12:35:44.561 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-04-28T12:35:44.561 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-04-28T12:35:44.561 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:35:44.780 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T12:35:44.827 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T12:35:44.828 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:44.828 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-28T12:35:44.828 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:35:44.911 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T12:35:44.940 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T12:35:45.085 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T12:35:45.264 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-28T12:35:45.301 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T12:35:45.368 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T12:35:45.368 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:45.368 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-28T12:35:45.368 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T12:35:45.368 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T12:35:45.368 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:35:45.427 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T12:35:45.464 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T12:35:45.632 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-28T12:35:45.718 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-28T12:35:46.465 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T12:35:46.541 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T12:35:46.541 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:46.541 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-28T12:35:46.541 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T12:35:46.541 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T12:35:46.542 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:35:46.730 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-28T12:35:46.765 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T12:35:46.790 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T12:35:46.831 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T12:35:46.831 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:46.831 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-28T12:35:46.831 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T12:35:46.831 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T12:35:46.831 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:35:46.880 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T12:35:46.880 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:46.880 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-28T12:35:46.880 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T12:35:46.880 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T12:35:46.880 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:35:46.881 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T12:35:46.948 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T12:35:46.948 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:46.948 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-28T12:35:46.948 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:35:47.031 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T12:35:47.105 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T12:35:47.105 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:47.105 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-28T12:35:47.105 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T12:35:47.105 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T12:35:47.105 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:35:47.366 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T12:35:47.432 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T12:35:47.432 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:47.432 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-28T12:35:47.432 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T12:35:47.432 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T12:35:47.432 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:35:47.499 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T12:35:47.558 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-28T12:35:47.566 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T12:35:47.566 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:47.566 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-28T12:35:47.566 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T12:35:47.566 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T12:35:47.566 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:35:47.619 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-28T12:35:47.700 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-28T12:35:47.937 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-28T12:35:47.995 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-28T12:35:48.079 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-28T12:35:48.189 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T12:35:48.346 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T12:35:48.408 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-28T12:35:48.452 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-28T12:35:48.485 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-28T12:35:49.121 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T12:35:49.164 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T12:35:49.221 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-28T12:35:49.327 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-28T12:35:49.357 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T12:35:50.664 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T12:35:50.709 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T12:35:51.808 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T12:35:51.834 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T12:35:51.979 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T12:35:52.155 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-28T12:35:52.186 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T12:35:52.254 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T12:35:52.255 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:52.255 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-28T12:35:52.255 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T12:35:52.255 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T12:35:52.255 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:35:52.312 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T12:35:52.349 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T12:35:52.461 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-28T12:35:52.519 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-28T12:35:52.611 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-28T12:35:52.667 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-28T12:35:52.752 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-28T12:35:52.801 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-28T12:35:52.836 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-28T12:35:53.166 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-28T12:35:53.214 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-28T12:35:53.242 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-28T12:35:53.512 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-28T12:35:53.548 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T12:35:53.614 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T12:35:53.614 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:53.614 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-28T12:35:53.614 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T12:35:53.614 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T12:35:53.614 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:35:53.665 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T12:35:53.733 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T12:35:53.733 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:53.733 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-28T12:35:53.733 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:35:53.824 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T12:35:53.899 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T12:35:53.899 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:53.899 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-28T12:35:53.899 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T12:35:53.899 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T12:35:53.900 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:35:54.159 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:35:54.216 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:35:54.250 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T12:35:54.273 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:35:54.323 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T12:35:54.323 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:54.323 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-28T12:35:54.323 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T12:35:54.323 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T12:35:54.323 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:35:55.651 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-28T12:35:55.757 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-28T12:35:55.864 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-28T12:35:56.058 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-28T12:35:56.105 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-28T12:35:56.106 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T12:35:56.234 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T12:35:56.274 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T12:35:56.274 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T12:35:56.413 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T12:35:56.533 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:35:56.814 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:35:56.814 INFO:teuthology.orchestra.run.smithi131.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-28T12:35:56.814 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:35:59.343 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-28T12:35:59.398 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-28T12:35:59.493 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-28T12:35:59.539 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-28T12:35:59.588 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-28T12:35:59.943 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-28T12:35:59.998 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-28T12:36:00.026 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-28T12:36:00.910 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:36:00.957 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:36:01.017 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:36:02.403 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-28T12:36:02.514 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-28T12:36:02.616 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-28T12:36:02.724 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:36:02.725 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-28T12:36:02.726 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-28T12:36:02.727 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-28T12:36:02.738 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-28T12:36:02.738 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-28T12:36:02.738 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-28T12:36:02.738 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-28T12:36:02.738 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-28T12:36:02.738 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-28T12:36:02.738 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-28T12:36:02.739 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-28T12:36:02.740 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-28T12:36:02.741 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-28T12:36:02.742 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-28T12:36:02.743 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-28T12:36:02.744 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-28T12:36:02.745 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-28T12:36:02.746 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-28T12:36:02.747 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-28T12:36:02.747 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-28T12:36:02.747 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-28T12:36:02.747 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-28T12:36:02.747 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-28T12:36:02.747 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-28T12:36:02.747 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-28T12:36:02.747 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T12:36:02.747 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-28T12:36:02.790 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-28T12:36:02.836 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-28T12:36:02.837 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T12:36:02.960 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T12:36:02.993 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T12:36:02.994 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T12:36:03.143 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T12:36:03.273 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:36:03.527 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:36:03.527 INFO:teuthology.orchestra.run.smithi157.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-28T12:36:03.527 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:36:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:05.166 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-04-28T12:36:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.166 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-28T12:36:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-28T12:36:05.167 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-04-28T12:36:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-28T12:36:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: automake-1.16.2-8.el9.noarch 2024-04-28T12:36:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: bison-3.7.4-5.el9.x86_64 2024-04-28T12:36:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-28T12:36:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.167 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.168 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: flex-2.6.4-9.el9.x86_64 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-28T12:36:05.169 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-28T12:36:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-28T12:36:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-28T12:36:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-28T12:36:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-28T12:36:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-28T12:36:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-28T12:36:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-28T12:36:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.170 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-28T12:36:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.173 INFO:teuthology.orchestra.run.smithi131.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: lua-5.4.4-4.el9.x86_64 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-13.14-1.el9.x86_64 2024-04-28T12:36:05.174 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-28T12:36:05.175 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-28T12:36:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-28T12:36:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-28T12:36:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-28T12:36:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-28T12:36:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-28T12:36:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-28T12:36:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-28T12:36:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-28T12:36:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-28T12:36:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-28T12:36:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-28T12:36:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply-3.11-14.el9.noarch 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-28T12:36:05.178 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-28T12:36:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-28T12:36:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-28T12:36:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-28T12:36:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-28T12:36:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-28T12:36:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-28T12:36:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-28T12:36:05.179 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: re2-1:20211101-3.el9.x86_64 2024-04-28T12:36:05.180 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-28T12:36:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-28T12:36:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-28T12:36:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-28T12:36:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-28T12:36:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-28T12:36:05.181 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:05.181 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-04-28T12:36:05.795 DEBUG:teuthology.parallel:result is None 2024-04-28T12:36:09.361 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-28T12:36:09.361 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-28T12:36:09.362 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-28T12:36:09.362 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-28T12:36:09.362 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-28T12:36:09.362 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-28T12:36:09.362 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-28T12:36:09.362 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:36:09.362 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:36:09.362 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T12:36:09.362 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-28T12:36:09.362 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-28T12:36:09.362 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:36:09.362 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-28T12:36:09.363 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-28T12:36:09.375 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-28T12:36:09.375 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-28T12:36:09.375 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-28T12:36:09.375 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-28T12:36:09.375 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-28T12:36:09.375 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-28T12:36:09.376 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-28T12:36:09.377 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-28T12:36:09.378 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-28T12:36:09.379 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-28T12:36:09.380 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-28T12:36:09.381 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-28T12:36:09.382 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-28T12:36:09.383 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-28T12:36:09.384 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-28T12:36:09.384 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-28T12:36:09.384 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T12:36:09.384 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-28T12:36:11.717 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:36:11.717 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:11.717 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-04-28T12:36:11.717 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.717 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.717 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-28T12:36:11.717 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-28T12:36:11.718 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-28T12:36:11.718 INFO:teuthology.orchestra.run.smithi157.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-28T12:36:11.718 INFO:teuthology.orchestra.run.smithi157.stdout: automake-1.16.2-8.el9.noarch 2024-04-28T12:36:11.718 INFO:teuthology.orchestra.run.smithi157.stdout: bison-3.7.4-5.el9.x86_64 2024-04-28T12:36:11.718 INFO:teuthology.orchestra.run.smithi157.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-28T12:36:11.718 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.718 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.718 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.718 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.718 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:11.718 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.718 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.719 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: flex-2.6.4-9.el9.x86_64 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-28T12:36:11.720 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.721 INFO:teuthology.orchestra.run.smithi157.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-28T12:36:11.722 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-28T12:36:11.722 INFO:teuthology.orchestra.run.smithi157.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-28T12:36:11.722 INFO:teuthology.orchestra.run.smithi157.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-28T12:36:11.722 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-28T12:36:11.722 INFO:teuthology.orchestra.run.smithi157.stdout: lua-5.4.4-4.el9.x86_64 2024-04-28T12:36:11.722 INFO:teuthology.orchestra.run.smithi157.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-28T12:36:11.722 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-28T12:36:11.722 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-28T12:36:11.722 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-28T12:36:11.722 INFO:teuthology.orchestra.run.smithi157.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-28T12:36:11.722 INFO:teuthology.orchestra.run.smithi157.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-13.14-1.el9.x86_64 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.723 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-28T12:36:11.724 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-28T12:36:11.725 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-28T12:36:11.726 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-28T12:36:11.727 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: re2-1:20211101-3.el9.x86_64 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-28T12:36:11.728 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-28T12:36:11.729 INFO:teuthology.orchestra.run.smithi157.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-28T12:36:11.729 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:11.729 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-28T12:36:12.325 DEBUG:teuthology.parallel:result is None 2024-04-28T12:36:12.326 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:36:12.581 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-28T12:36:12.641 INFO:teuthology.orchestra.run.smithi131.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-28T12:36:12.641 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-28T12:36:12.642 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-28T12:36:12.644 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:36:12.864 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-28T12:36:12.930 INFO:teuthology.orchestra.run.smithi157.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-28T12:36:12.930 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-28T12:36:12.930 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-28T12:36:12.932 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-28T12:36:12.932 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:12.933 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-28T12:36:13.023 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:13.023 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-28T12:36:13.100 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-28T12:36:13.101 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:13.101 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-04-28T12:36:13.186 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-28T12:36:13.272 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:13.272 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-04-28T12:36:13.342 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-28T12:36:13.451 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-28T12:36:13.451 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:13.451 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-28T12:36:13.520 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-28T12:36:13.634 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:13.635 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-28T12:36:13.707 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-28T12:36:13.811 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-28T12:36:13.811 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:13.811 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-04-28T12:36:13.882 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-28T12:36:13.989 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:13.990 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-04-28T12:36:14.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-28T12:36:14.165 INFO:teuthology.run_tasks:Running task ceph... 2024-04-28T12:36:14.248 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-28T12:36:14.248 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /var/log/ceph 2024-04-28T12:36:14.253 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2024-04-28T12:36:14.321 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-28T12:36:14.321 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-28T12:36:14.326 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-28T12:36:14.398 INFO:tasks.ceph:Creating extra log directories... 2024-04-28T12:36:14.398 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-28T12:36:14.402 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-28T12:36:14.476 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-28T12:36:14.476 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-28T12:36:14.477 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-28T12:36:14.477 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:14.477 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-28T12:36:14.518 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-28T12:36:14.695 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-28T12:36:14.764 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:14.764 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-28T12:36:14.806 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-28T12:36:14.992 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-28T12:36:15.066 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-28T12:36:15.066 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-28T12:36:15.066 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls}', 'email': None, 'first_in_suite': False, 'job_id': '7676878', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi131.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNGKrQx3beiFGGGKr1IgBUo3gSRENd3cGQgZpEJ851txl5x6woWTKky3Fo7G49I7uRXQviEpZe6QxuTI98CKcvw=', 'smithi157.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAKcrhOvz6kpvYZgifp4WCPpwgGaA25tfLv08kMoYPpogHbOyuqFYVN/ZfdWJX+EUTPcB4G/dIKH8aVQiM70axQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_acls']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-28_11:41:23', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-28T12:36:15.067 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-28T12:36:15.072 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-28T12:36:15.117 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m0777 -- /var/run/ceph 2024-04-28T12:36:15.122 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2024-04-28T12:36:15.212 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:15.212 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-04-28T12:36:15.259 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-28T12:36:15.260 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-04-28T12:36:15.350 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-28T12:36:15.350 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:36:15.350 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-28T12:36:15.350 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:36:15.350 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:36:15.350 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-04-28 12:35:57.300328654 +0000 2024-04-28T12:36:15.350 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-04-28 12:31:40.469766779 +0000 2024-04-28T12:36:15.350 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-04-28 12:31:40.469766779 +0000 2024-04-28T12:36:15.351 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-04-28 12:31:40.469766779 +0000 2024-04-28T12:36:15.351 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-28T12:36:15.422 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-04-28T12:36:15.423 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-04-28T12:36:15.423 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000513639 s, 997 kB/s 2024-04-28T12:36:15.426 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-28T12:36:15.472 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-04-28T12:36:15.517 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-28T12:36:15.517 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:36:15.517 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-28T12:36:15.517 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:36:15.517 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:36:15.518 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-04-28 12:35:57.301328633 +0000 2024-04-28T12:36:15.518 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-04-28 12:31:41.051754611 +0000 2024-04-28T12:36:15.518 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-04-28 12:31:41.051754611 +0000 2024-04-28T12:36:15.518 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-04-28 12:31:41.051754611 +0000 2024-04-28T12:36:15.518 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-28T12:36:15.639 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-04-28T12:36:15.639 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-04-28T12:36:15.639 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000382205 s, 1.3 MB/s 2024-04-28T12:36:15.643 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-28T12:36:15.695 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-04-28T12:36:15.743 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-28T12:36:15.743 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:36:15.743 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-28T12:36:15.743 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:36:15.743 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:36:15.743 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-04-28 12:35:57.302328612 +0000 2024-04-28T12:36:15.743 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-04-28 12:31:41.644742213 +0000 2024-04-28T12:36:15.743 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-04-28 12:31:41.644742213 +0000 2024-04-28T12:36:15.743 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-04-28 12:31:41.644742213 +0000 2024-04-28T12:36:15.744 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-28T12:36:15.859 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-04-28T12:36:15.859 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-04-28T12:36:15.859 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000352142 s, 1.5 MB/s 2024-04-28T12:36:15.862 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-28T12:36:15.916 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-04-28T12:36:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-28T12:36:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:36:15.964 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-28T12:36:15.964 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:36:15.964 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:36:15.964 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-04-28 12:35:57.303328591 +0000 2024-04-28T12:36:15.964 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-04-28 12:31:42.227730024 +0000 2024-04-28T12:36:15.964 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-04-28 12:31:42.227730024 +0000 2024-04-28T12:36:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-04-28 12:31:42.227730024 +0000 2024-04-28T12:36:15.965 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-28T12:36:16.040 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-04-28T12:36:16.041 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-04-28T12:36:16.041 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000418808 s, 1.2 MB/s 2024-04-28T12:36:16.044 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-28T12:36:16.100 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-28T12:36:16.101 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:16.101 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-04-28T12:36:16.145 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-28T12:36:16.146 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-04-28T12:36:16.222 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-28T12:36:16.223 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:36:16.223 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-28T12:36:16.223 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:36:16.223 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:36:16.223 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-28 12:36:04.041038026 +0000 2024-04-28T12:36:16.223 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-28 12:31:40.470698407 +0000 2024-04-28T12:36:16.223 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-28 12:31:40.470698407 +0000 2024-04-28T12:36:16.223 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-28 12:31:40.470698407 +0000 2024-04-28T12:36:16.223 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-28T12:36:16.327 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-28T12:36:16.328 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-28T12:36:16.328 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000301112 s, 1.7 MB/s 2024-04-28T12:36:16.331 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-28T12:36:16.378 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-04-28T12:36:16.421 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-28T12:36:16.421 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:36:16.421 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-28T12:36:16.421 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:36:16.421 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:36:16.421 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-28 12:36:04.042038008 +0000 2024-04-28T12:36:16.421 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-28 12:31:41.029688536 +0000 2024-04-28T12:36:16.421 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-28 12:31:41.029688536 +0000 2024-04-28T12:36:16.422 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-28 12:31:41.029688536 +0000 2024-04-28T12:36:16.422 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-28T12:36:16.521 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-28T12:36:16.521 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-28T12:36:16.521 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.00017835 s, 2.9 MB/s 2024-04-28T12:36:16.525 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-28T12:36:16.564 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-04-28T12:36:16.643 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-28T12:36:16.643 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:36:16.643 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-28T12:36:16.643 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:36:16.643 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:36:16.644 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-28 12:36:04.042038008 +0000 2024-04-28T12:36:16.644 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-28 12:31:41.629677941 +0000 2024-04-28T12:36:16.644 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-28 12:31:41.629677941 +0000 2024-04-28T12:36:16.644 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-28 12:31:41.629677941 +0000 2024-04-28T12:36:16.644 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-28T12:36:16.747 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-28T12:36:16.747 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-28T12:36:16.747 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000175789 s, 2.9 MB/s 2024-04-28T12:36:16.751 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-28T12:36:16.791 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-04-28T12:36:16.872 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-28T12:36:16.872 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:36:16.872 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-28T12:36:16.872 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:36:16.873 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:36:16.873 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-28 12:36:04.043037990 +0000 2024-04-28T12:36:16.873 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-28 12:31:42.230667328 +0000 2024-04-28T12:36:16.873 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-28 12:31:42.230667328 +0000 2024-04-28T12:36:16.873 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-28 12:31:42.230667328 +0000 2024-04-28T12:36:16.873 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-28T12:36:16.971 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-28T12:36:16.971 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-28T12:36:16.971 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000211034 s, 2.4 MB/s 2024-04-28T12:36:16.975 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-28T12:36:17.016 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-28T12:36:17.016 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-28T12:36:17.016 INFO:tasks.ceph:Generating config... 2024-04-28T12:36:17.018 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-28T12:36:17.018 INFO:tasks.ceph:[client] debug client = 20 2024-04-28T12:36:17.018 INFO:tasks.ceph:[client] debug ms = 1 2024-04-28T12:36:17.018 INFO:tasks.ceph:[client] pid file = 2024-04-28T12:36:17.018 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-28T12:36:17.018 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-28T12:36:17.018 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-28T12:36:17.018 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-28T12:36:17.018 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-28T12:36:17.018 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-28T12:36:17.019 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-28T12:36:17.020 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-28T12:36:17.021 INFO:tasks.ceph:Setting up mon.a... 2024-04-28T12:36:17.021 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-28T12:36:17.145 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.keyring 2024-04-28T12:36:17.153 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-28T12:36:17.277 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-28T12:36:17.360 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.131'), ('mon.b', '172.21.15.157'), ('mon.c', '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]')] 2024-04-28T12:36:17.360 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.131,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-28T12:36:17.361 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:17.361 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-28T12:36:17.413 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.131 --add b 172.21.15.157 --addv c '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:17.568 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:17.568 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: generated fsid fc0efa02-d382-4f6a-947e-cbe7ab2d58d1 2024-04-28T12:36:17.568 INFO:teuthology.orchestra.run.smithi131.stdout:setting min_mon_release = quincy 2024-04-28T12:36:17.568 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 0 2024-04-28T12:36:17.568 INFO:teuthology.orchestra.run.smithi131.stdout:fsid fc0efa02-d382-4f6a-947e-cbe7ab2d58d1 2024-04-28T12:36:17.568 INFO:teuthology.orchestra.run.smithi131.stdout:last_changed 2024-04-28T12:36:17.572247+0000 2024-04-28T12:36:17.568 INFO:teuthology.orchestra.run.smithi131.stdout:created 2024-04-28T12:36:17.572247+0000 2024-04-28T12:36:17.568 INFO:teuthology.orchestra.run.smithi131.stdout:min_mon_release 17 (quincy) 2024-04-28T12:36:17.568 INFO:teuthology.orchestra.run.smithi131.stdout:election_strategy: 1 2024-04-28T12:36:17.569 INFO:teuthology.orchestra.run.smithi131.stdout:0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2024-04-28T12:36:17.569 INFO:teuthology.orchestra.run.smithi131.stdout:1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.b 2024-04-28T12:36:17.569 INFO:teuthology.orchestra.run.smithi131.stdout:2: [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] mon.c 2024-04-28T12:36:17.569 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-28T12:36:17.571 DEBUG:teuthology.orchestra.run.smithi131:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-28T12:36:17.619 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fc0efa02-d382-4f6a-947e-cbe7ab2d58d1... 2024-04-28T12:36:17.621 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-28T12:36:17.626 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-28T12:36:17.738 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2024-04-28T12:36:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2024-04-28T12:36:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-28T12:36:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2024-04-28T12:36:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2024-04-28T12:36:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2024-04-28T12:36:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2024-04-28T12:36:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2024-04-28T12:36:17.739 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2024-04-28T12:36:17.740 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2024-04-28T12:36:17.741 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.131,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = fc0efa02-d382-4f6a-947e-cbe7ab2d58d1 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2024-04-28T12:36:17.742 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug trim objects = true 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.743 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2024-04-28T12:36:17.744 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2024-04-28T12:36:17.744 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.744 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2024-04-28T12:36:17.744 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2024-04-28T12:36:17.744 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2024-04-28T12:36:17.744 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2024-04-28T12:36:17.744 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 20 2024-04-28T12:36:17.744 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 20 2024-04-28T12:36:17.744 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-28T12:36:17.744 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2024-04-28T12:36:17.744 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 10 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-04-28T12:36:17.745 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-04-28T12:36:17.746 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-04-28T12:36:17.746 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2024-04-28T12:36:17.746 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-04-28T12:36:17.746 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.746 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2024-04-28T12:36:17.746 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-28T12:36:17.746 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-04-28T12:36:17.746 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2024-04-28T12:36:17.746 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-04-28T12:36:17.746 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2024-04-28T12:36:17.746 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2024-04-28T12:36:17.746 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2024-04-28T12:36:17.747 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: mon down mkfs grace = 300 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: mon op complaint time = 120 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: client mount timeout = 600 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2024-04-28T12:36:17.748 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout:[mds] 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds = 20 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout: debug mds balancer = 20 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug frag = True 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout: mds debug scatterstat = True 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout: mds op complaint time = 180 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout: mds verify scatter = True 2024-04-28T12:36:17.749 INFO:teuthology.orchestra.run.smithi157.stdout: osd op complaint time = 180 2024-04-28T12:36:17.750 INFO:teuthology.orchestra.run.smithi157.stdout: rados mon op timeout = 900 2024-04-28T12:36:17.750 INFO:teuthology.orchestra.run.smithi157.stdout: rados osd op timeout = 900 2024-04-28T12:36:17.758 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: chdir = "" 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: auth supported = cephx 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: filestore xattr use omap = true 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: mon clock drift allowed = 1.000 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: osd crush chooseleaf type = 0 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: auth debug = true 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on old message = true 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on bug = true 2024-04-28T12:36:17.759 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: mon pg warn max object skew = 0 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: # disable pg_autoscaler by default for new pools 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default size = 2 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow primary affinity = true 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow pg remap = true 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on legacy crush tunables = false 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on crush straw calc version zero = false 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on no sortbitwise = false 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on osd down out interval zero = false 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on too few osds = false 2024-04-28T12:36:17.760 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: mon_allow_pool_size_one = true 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: osd default data pool replay window = 5 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: mon allow pool delete = true 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: mon cluster log file level = debug 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: debug asserts on shutdown = true 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: mon health detail to clog = false 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: mon host = "172.21.15.131,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = fc0efa02-d382-4f6a-947e-cbe7ab2d58d1 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.761 INFO:teuthology.orchestra.run.smithi131.stdout:[osd] 2024-04-28T12:36:17.762 INFO:teuthology.orchestra.run.smithi131.stdout: osd journal size = 100 2024-04-28T12:36:17.762 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.762 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub load threshold = 5.0 2024-04-28T12:36:17.762 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub max interval = 600 2024-04-28T12:36:17.762 INFO:teuthology.orchestra.run.smithi131.stdout: osd mclock profile = high_recovery_ops 2024-04-28T12:36:17.762 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.762 INFO:teuthology.orchestra.run.smithi131.stdout: osd recover clone overlap = true 2024-04-28T12:36:17.762 INFO:teuthology.orchestra.run.smithi131.stdout: osd recovery max chunk = 1048576 2024-04-28T12:36:17.762 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.762 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug shutdown = true 2024-04-28T12:36:17.762 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug op order = true 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug verify stray on activate = true 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug trim objects = true 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: osd open classes on start = true 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug pg log writeout = true 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: osd deep scrub update digest min age = 30 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: osd map max advance = 10 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: journal zero on create = true 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: filestore ondisk finisher threads = 3 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: filestore apply finisher threads = 3 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: bdev debug aio = true 2024-04-28T12:36:17.763 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug misdirected ops = true 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore block size = 96636764160 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore fsck on mount = True 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluefs = 20 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluestore = 20 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: debug osd = 20 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: debug rocksdb = 10 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd backfillfull_ratio = 0.85 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd full ratio = 0.9 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd nearfull ratio = 0.8 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: osd failsafe full ratio = 0.95 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: osd objectstore = bluestore 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: osd op complaint time = 180 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr] 2024-04-28T12:36:17.764 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: debug mgr = 20 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: mgr/telemetry/nag = false 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: debug client = 20 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout:[mon] 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: debug paxos = 20 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: mon data avail warn = 5 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: mon mgr mkfs grace = 240 2024-04-28T12:36:17.765 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd reporter subtree level = osd 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd prime pg temp = true 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: auth mon ticket ttl = 660 # 11m 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: auth service ticket ttl = 240 # 4m 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: # don't complain about insecure global_id in the test suite 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: # 1m isn't quite enough 2024-04-28T12:36:17.766 INFO:teuthology.orchestra.run.smithi131.stdout: mon_down_mkfs_grace = 2m 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_filestore_osds = false 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: mon down mkfs grace = 300 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: mon op complaint time = 120 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout:[client] 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: rgw cache enabled = true 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable ops log = true 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable usage log = true 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: client mount timeout = 600 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: debug client = 20 2024-04-28T12:36:17.767 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = "" 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout: rados mon op timeout = 900 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout: rados osd op timeout = 900 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.a] 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.b] 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.c] 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout:[mds] 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout: debug mds = 20 2024-04-28T12:36:17.768 INFO:teuthology.orchestra.run.smithi131.stdout: debug mds balancer = 20 2024-04-28T12:36:17.769 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-04-28T12:36:17.769 INFO:teuthology.orchestra.run.smithi131.stdout: mds debug frag = True 2024-04-28T12:36:17.769 INFO:teuthology.orchestra.run.smithi131.stdout: mds debug scatterstat = True 2024-04-28T12:36:17.769 INFO:teuthology.orchestra.run.smithi131.stdout: mds op complaint time = 180 2024-04-28T12:36:17.769 INFO:teuthology.orchestra.run.smithi131.stdout: mds verify scatter = True 2024-04-28T12:36:17.769 INFO:teuthology.orchestra.run.smithi131.stdout: osd op complaint time = 180 2024-04-28T12:36:17.769 INFO:teuthology.orchestra.run.smithi131.stdout: rados mon op timeout = 900 2024-04-28T12:36:17.769 INFO:teuthology.orchestra.run.smithi131.stdout: rados osd op timeout = 900 2024-04-28T12:36:17.792 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-28T12:36:17.792 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-28T12:36:17.917 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-28T12:36:17.917 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:17.917 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-28T12:36:17.966 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:17.966 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-28T12:36:18.061 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:36:18.061 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:18.061 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-28T12:36:18.061 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-28T12:36:18.173 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:18.173 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:18.270 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:36:18.270 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:18.270 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-28T12:36:18.270 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-28T12:36:18.364 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:18.364 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:18.443 INFO:tasks.ceph:Setting up mon nodes... 2024-04-28T12:36:18.443 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-28T12:36:18.443 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-28T12:36:18.610 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-28T12:36:18.620 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-28T12:36:18.791 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-28T12:36:18.799 INFO:tasks.ceph:Setting up mds nodes... 2024-04-28T12:36:18.800 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-28T12:36:18.949 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-28T12:36:18.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-28T12:36:19.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-28T12:36:19.207 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-28T12:36:19.216 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-28T12:36:19.299 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-28T12:36:19.455 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-28T12:36:19.461 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-28T12:36:19.528 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-28T12:36:19.648 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-28T12:36:19.655 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-28T12:36:19.731 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-28T12:36:19.732 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-28T12:36:19.855 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-28T12:36:19.895 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-28T12:36:20.012 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-28T12:36:20.048 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-28T12:36:20.166 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-28T12:36:20.204 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-28T12:36:20.321 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-28T12:36:20.356 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-28T12:36:20.356 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-28T12:36:20.356 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-28T12:36:20.444 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-28T12:36:20.444 INFO:tasks.ceph:role: osd.0 2024-04-28T12:36:20.444 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:36:20.444 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-28T12:36:20.534 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T12:36:20.534 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T12:36:20.534 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T12:36:20.534 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T12:36:20.534 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T12:36:20.535 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-04-28T12:36:20.535 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T12:36:20.535 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T12:36:20.535 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T12:36:20.535 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T12:36:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-04-28T12:36:20.956 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-04-28T12:36:20.956 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-28T12:36:21.127 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-28T12:36:21.214 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-28T12:36:21.288 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-28T12:36:21.288 INFO:tasks.ceph:role: osd.1 2024-04-28T12:36:21.288 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:36:21.288 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-28T12:36:21.375 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T12:36:21.376 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T12:36:21.376 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T12:36:21.376 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T12:36:21.376 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T12:36:21.376 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-04-28T12:36:21.376 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T12:36:21.376 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T12:36:21.376 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T12:36:21.376 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T12:36:21.731 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-04-28T12:36:21.737 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-04-28T12:36:21.737 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-28T12:36:21.859 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-28T12:36:21.938 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-28T12:36:22.009 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-28T12:36:22.009 INFO:tasks.ceph:role: osd.2 2024-04-28T12:36:22.009 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:36:22.009 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-28T12:36:22.094 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T12:36:22.094 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T12:36:22.094 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T12:36:22.094 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T12:36:22.094 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T12:36:22.094 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-04-28T12:36:22.094 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T12:36:22.094 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T12:36:22.095 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T12:36:22.095 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T12:36:22.493 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-04-28T12:36:22.499 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-04-28T12:36:22.499 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-28T12:36:22.649 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-28T12:36:22.738 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-28T12:36:22.816 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-28T12:36:22.816 INFO:tasks.ceph:role: osd.3 2024-04-28T12:36:22.816 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:36:22.816 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-28T12:36:22.901 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T12:36:22.901 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T12:36:22.901 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T12:36:22.902 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T12:36:22.902 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T12:36:22.902 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-04-28T12:36:22.902 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T12:36:22.902 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T12:36:22.902 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T12:36:22.902 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T12:36:23.310 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-04-28T12:36:23.315 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-04-28T12:36:23.316 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-28T12:36:23.449 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-28T12:36:23.535 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:23.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:23.678+0000 7fc7e9155640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-28T12:36:23.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:23.678+0000 7fc7e9155640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-28T12:36:23.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:23.679+0000 7fc7e9155640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-28T12:36:23.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:23.679+0000 7fc7e9155640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-28T12:36:23.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:23.679+0000 7fc7e9155640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-28T12:36:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:23.683+0000 7fc7e9155640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-28T12:36:26.637 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-28T12:36:26.722 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:26.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:26.855+0000 7f692a40d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-28T12:36:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:26.856+0000 7f692a40d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-28T12:36:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:26.856+0000 7f692a40d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-28T12:36:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:26.856+0000 7f692a40d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-28T12:36:26.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:26.856+0000 7f692a40d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-28T12:36:26.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:26.859+0000 7f692a40d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-28T12:36:30.088 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-28T12:36:30.174 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:30.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:30.303+0000 7fc40a624640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-28T12:36:30.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:30.304+0000 7fc40a624640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-28T12:36:30.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:30.304+0000 7fc40a624640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-28T12:36:30.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:30.304+0000 7fc40a624640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-28T12:36:30.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:30.304+0000 7fc40a624640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-28T12:36:30.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:30.306+0000 7fc40a624640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-28T12:36:33.532 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-28T12:36:33.617 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:33.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:33.749+0000 7fca13c43640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-28T12:36:33.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:33.750+0000 7fca13c43640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-28T12:36:33.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:33.750+0000 7fca13c43640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-28T12:36:33.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:33.750+0000 7fca13c43640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-28T12:36:33.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:33.750+0000 7fca13c43640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-28T12:36:33.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:33.752+0000 7fca13c43640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-28T12:36:36.947 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-28T12:36:37.030 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-28T12:36:37.120 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-28T12:36:37.120 INFO:tasks.ceph:role: osd.4 2024-04-28T12:36:37.120 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:36:37.120 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-28T12:36:37.202 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T12:36:37.203 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T12:36:37.203 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T12:36:37.203 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T12:36:37.203 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T12:36:37.203 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-28T12:36:37.203 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T12:36:37.203 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T12:36:37.203 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T12:36:37.203 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T12:36:37.239 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-28T12:36:37.245 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-28T12:36:37.245 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-28T12:36:37.342 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-28T12:36:37.423 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-28T12:36:37.491 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-28T12:36:37.492 INFO:tasks.ceph:role: osd.5 2024-04-28T12:36:37.492 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:36:37.492 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-28T12:36:37.566 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T12:36:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T12:36:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T12:36:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T12:36:37.566 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T12:36:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-28T12:36:37.566 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T12:36:37.566 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T12:36:37.566 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T12:36:37.566 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T12:36:37.602 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-28T12:36:37.607 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-28T12:36:37.608 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-28T12:36:37.709 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-28T12:36:37.782 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-28T12:36:37.853 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-28T12:36:37.854 INFO:tasks.ceph:role: osd.6 2024-04-28T12:36:37.854 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:36:37.854 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-28T12:36:37.930 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T12:36:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T12:36:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T12:36:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T12:36:37.930 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T12:36:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-28T12:36:37.930 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T12:36:37.930 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T12:36:37.930 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T12:36:37.931 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T12:36:37.964 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-28T12:36:37.969 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-28T12:36:37.969 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-28T12:36:38.067 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-28T12:36:38.137 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-28T12:36:38.208 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-28T12:36:38.208 INFO:tasks.ceph:role: osd.7 2024-04-28T12:36:38.208 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:36:38.209 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-28T12:36:38.286 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-28T12:36:38.286 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-28T12:36:38.286 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-28T12:36:38.286 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-28T12:36:38.286 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-28T12:36:38.286 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-04-28T12:36:38.287 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-28T12:36:38.287 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-28T12:36:38.287 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-28T12:36:38.287 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-28T12:36:38.320 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-04-28T12:36:38.325 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-04-28T12:36:38.325 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-28T12:36:38.431 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-28T12:36:38.506 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:38.638 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:38.641+0000 7fd8dc5a7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-28T12:36:38.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:38.642+0000 7fd8dc5a7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-28T12:36:38.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:38.642+0000 7fd8dc5a7640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-28T12:36:38.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:38.642+0000 7fd8dc5a7640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-28T12:36:38.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:38.642+0000 7fd8dc5a7640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-28T12:36:38.642 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:38.645+0000 7fd8dc5a7640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-28T12:36:41.616 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-28T12:36:41.702 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:41.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:41.832+0000 7fc5ed531640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-28T12:36:41.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:41.832+0000 7fc5ed531640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-28T12:36:41.829 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:41.832+0000 7fc5ed531640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-28T12:36:41.830 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:41.832+0000 7fc5ed531640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-28T12:36:41.830 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:41.832+0000 7fc5ed531640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-28T12:36:41.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:41.835+0000 7fc5ed531640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-28T12:36:44.788 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-28T12:36:44.872 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:44.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:44.994+0000 7fbeb2a41640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-28T12:36:44.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:44.994+0000 7fbeb2a41640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-28T12:36:44.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:44.994+0000 7fbeb2a41640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-28T12:36:44.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:44.994+0000 7fbeb2a41640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-28T12:36:44.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:44.995+0000 7fbeb2a41640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-28T12:36:44.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:44.997+0000 7fbeb2a41640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-28T12:36:45.068 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:36:45.076 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:36:47.971 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-28T12:36:48.050 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:48.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:48.178+0000 7f364a420640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-28T12:36:48.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:48.179+0000 7f364a420640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-28T12:36:48.177 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:48.179+0000 7f364a420640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-28T12:36:48.177 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:48.179+0000 7f364a420640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-28T12:36:48.177 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:48.179+0000 7f364a420640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-28T12:36:48.178 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:36:48.181+0000 7f364a420640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-28T12:36:51.404 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-28T12:36:51.491 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-28T12:36:51.492 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:51.492 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-28T12:36:51.581 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:51.581 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-28T12:36:51.661 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:51.661 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-28T12:36:51.744 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:51.745 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-28T12:36:51.874 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:51.875 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-28T12:36:51.957 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:51.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-28T12:36:52.043 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:52.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-28T12:36:52.171 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:52.171 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-28T12:36:52.259 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:52.259 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-28T12:36:52.341 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:52.341 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-28T12:36:52.425 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:52.425 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-28T12:36:52.509 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:52.509 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-28T12:36:52.589 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:52.589 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-28T12:36:52.673 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:52.673 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-28T12:36:52.755 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:52.755 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-28T12:36:52.813 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:36:52.814 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-28T12:36:52.867 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:52.867 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-28T12:36:52.924 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:36:52.924 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-28T12:36:52.975 INFO:tasks.ceph:Adding keys to all mons... 2024-04-28T12:36:52.975 DEBUG:teuthology.orchestra.run.smithi131:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-28T12:36:52.982 DEBUG:teuthology.orchestra.run.smithi157:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-28T12:36:53.052 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.y] 2024-04-28T12:36:53.052 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDCQi5mPkqdJBAAYU06FiAGyuWtxBTcwsXUQg== 2024-04-28T12:36:53.052 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-04-28T12:36:53.052 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDCQi5mAK7dOBAAPuF0VSQ8/oA/TabJenHHTA== 2024-04-28T12:36:53.052 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-04-28T12:36:53.052 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDDQi5mn6mlDBAA4YLF/yZrzfJVuw7NuYNE6g== 2024-04-28T12:36:53.052 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.0] 2024-04-28T12:36:53.052 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDHQi5m1zB9KBAAA4XL4E3Zimv7IfK8gIwlog== 2024-04-28T12:36:53.052 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.1] 2024-04-28T12:36:53.052 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDKQi5mCqcTMxAAp2DAM3sQFnWvE7tUmDCsgg== 2024-04-28T12:36:53.052 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.2] 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDOQi5mVYspEhAAIAcbyMkwxxHmRB2rTnYBZQ== 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.3] 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDRQi5mcOO/LBAACgiwbkFt6BTFeXDfh0RtsQ== 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.x] 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDCQi5mlcRqLxAArJ9SXw5eNDzPr0nrWHUaPw== 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDDQi5my+1kGxAAQYO+yf04zcg7fflJbD6cMg== 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDDQi5mejXpJhAAKu1RVfnAQb9/dZLira9nAA== 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.4] 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDWQi5mPxxOJhAAHZqlRUkEKZUvtitkoULnAg== 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.5] 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDZQi5mRgeuMRAAlPWR+At5yl/wIrWFvEtFRQ== 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.6] 2024-04-28T12:36:53.053 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDcQi5mKH1ZOxAA+aQ7qRfEOfhAzgSjx3UQYg== 2024-04-28T12:36:53.054 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.7] 2024-04-28T12:36:53.054 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDgQi5mwlm9ChAA9oikwI0iUbxqbGXjXLZflw== 2024-04-28T12:36:53.054 INFO:teuthology.orchestra.run.smithi157.stdout:[client.0] 2024-04-28T12:36:53.054 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDDQi5mElI+MxAAW17tKD+nhz2A3X8a4qZmbw== 2024-04-28T12:36:53.054 INFO:teuthology.orchestra.run.smithi157.stdout:[client.2] 2024-04-28T12:36:53.054 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDEQi5mSa75ABAAGkSnVFXcrqSGrH11c1Yy6w== 2024-04-28T12:36:53.054 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2024-04-28T12:36:53.054 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDEQi5mwR0tChAAJTudcvUJmCRLI5oijLUqOw== 2024-04-28T12:36:53.054 INFO:teuthology.orchestra.run.smithi157.stdout:[client.3] 2024-04-28T12:36:53.054 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDEQi5mBMtmExAAe8U6grexkKUjuwlLlKN/wg== 2024-04-28T12:36:53.056 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr.y] 2024-04-28T12:36:53.056 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDCQi5mPkqdJBAAYU06FiAGyuWtxBTcwsXUQg== 2024-04-28T12:36:53.056 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2024-04-28T12:36:53.056 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDCQi5mAK7dOBAAPuF0VSQ8/oA/TabJenHHTA== 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDDQi5mn6mlDBAA4YLF/yZrzfJVuw7NuYNE6g== 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.0] 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDHQi5m1zB9KBAAA4XL4E3Zimv7IfK8gIwlog== 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.1] 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDKQi5mCqcTMxAAp2DAM3sQFnWvE7tUmDCsgg== 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.2] 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDOQi5mVYspEhAAIAcbyMkwxxHmRB2rTnYBZQ== 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.3] 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDRQi5mcOO/LBAACgiwbkFt6BTFeXDfh0RtsQ== 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr.x] 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDCQi5mlcRqLxAArJ9SXw5eNDzPr0nrWHUaPw== 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2024-04-28T12:36:53.057 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDDQi5my+1kGxAAQYO+yf04zcg7fflJbD6cMg== 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDDQi5mejXpJhAAKu1RVfnAQb9/dZLira9nAA== 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.4] 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDWQi5mPxxOJhAAHZqlRUkEKZUvtitkoULnAg== 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.5] 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDZQi5mRgeuMRAAlPWR+At5yl/wIrWFvEtFRQ== 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.6] 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDcQi5mKH1ZOxAA+aQ7qRfEOfhAzgSjx3UQYg== 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.7] 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDgQi5mwlm9ChAA9oikwI0iUbxqbGXjXLZflw== 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDDQi5mElI+MxAAW17tKD+nhz2A3X8a4qZmbw== 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout:[client.2] 2024-04-28T12:36:53.058 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDEQi5mSa75ABAAGkSnVFXcrqSGrH11c1Yy6w== 2024-04-28T12:36:53.059 INFO:teuthology.orchestra.run.smithi131.stdout:[client.1] 2024-04-28T12:36:53.059 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDEQi5mwR0tChAAJTudcvUJmCRLI5oijLUqOw== 2024-04-28T12:36:53.059 INFO:teuthology.orchestra.run.smithi131.stdout:[client.3] 2024-04-28T12:36:53.059 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDEQi5mBMtmExAAe8U6grexkKUjuwlLlKN/wg== 2024-04-28T12:36:53.061 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-28T12:36:53.110 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-28T12:36:53.247 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T12:36:53.255 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T12:36:53.394 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T12:36:53.399 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T12:36:53.538 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:53.545 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:53.678 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:53.686 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:53.821 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:53.827 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:53.956 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:53.963 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:54.097 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-28T12:36:54.105 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-28T12:36:54.238 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T12:36:54.245 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T12:36:54.385 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T12:36:54.390 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-28T12:36:54.519 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:54.523 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:54.657 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:54.664 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:54.795 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:54.802 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:54.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:54.942 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-28T12:36:55.075 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T12:36:55.082 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T12:36:55.218 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T12:36:55.225 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T12:36:55.364 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T12:36:55.371 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T12:36:55.512 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T12:36:55.522 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-28T12:36:55.651 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-28T12:36:55.652 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-28T12:36:55.739 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-28T12:36:55.892 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-28T12:36:55.963 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-28T12:36:56.053 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-28T12:36:56.199 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-28T12:36:56.264 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-28T12:36:56.337 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-28T12:36:56.481 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-28T12:36:56.552 DEBUG:teuthology.orchestra.run.smithi131:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:56.560 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-28T12:36:56.612 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-28T12:36:56.612 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-28T12:36:56.612 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-28T12:36:56.617 INFO:tasks.ceph.mon.a:Started 2024-04-28T12:36:56.617 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-28T12:36:56.617 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-28T12:36:56.622 INFO:tasks.ceph.mon.b:Started 2024-04-28T12:36:56.622 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-28T12:36:56.622 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-28T12:36:56.627 INFO:tasks.ceph.mon.c:Started 2024-04-28T12:36:56.627 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-28T12:36:56.627 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-28T12:36:56.627 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-28T12:36:56.632 INFO:tasks.ceph.mgr.y:Started 2024-04-28T12:36:56.632 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-28T12:36:56.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-28T12:36:56.636 INFO:tasks.ceph.mgr.x:Started 2024-04-28T12:36:56.636 INFO:tasks.ceph:Setting crush tunables to default 2024-04-28T12:36:56.636 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph osd crush tunables default 2024-04-28T12:36:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.814+0000 7faf1ce00640 1 Processor -- start 2024-04-28T12:36:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.814+0000 7faf1ce00640 1 -- start start 2024-04-28T12:36:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.815+0000 7faf1ce00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:36:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.815+0000 7faf1ce00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf180d2d70 con 0x7faf1815f180 2024-04-28T12:36:56.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.815+0000 7faf1ce00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf180d06e0 con 0x7faf180d3830 2024-04-28T12:36:56.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.815+0000 7faf1ce00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faf1815f560 con 0x7faf180b3cb0 2024-04-28T12:36:56.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.815+0000 7faf17400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7faf1815f180 0x7faf180d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40934/0 (socket says 172.21.15.131:40934) 2024-04-28T12:36:56.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.815+0000 7faf17400640 1 -- 172.21.15.131:0/971078238 learned_addr learned my addr 172.21.15.131:0/971078238 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:36:56.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.815+0000 7faf17e00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:36:56.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.816+0000 7faf17e00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 msgr2=0x7faf180d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T12:36:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.816+0000 7faf17e00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 msgr2=0x7faf180d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:36:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.816+0000 7faf17e00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:36:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:56.816+0000 7faf17e00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:36:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:57.017+0000 7faf17e00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:36:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:57.018+0000 7faf17e00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 msgr2=0x7faf180d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T12:36:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:57.018+0000 7faf17e00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 msgr2=0x7faf180d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:36:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:57.018+0000 7faf17e00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:36:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:57.018+0000 7faf17e00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-28T12:36:57.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:57.420+0000 7faf17e00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:36:57.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:57.420+0000 7faf17e00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 msgr2=0x7faf180d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T12:36:57.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:57.420+0000 7faf17e00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 msgr2=0x7faf180d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:36:57.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:57.420+0000 7faf17e00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:36:57.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:57.420+0000 7faf17e00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-28T12:36:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:58.223+0000 7faf17e00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:36:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:58.223+0000 7faf17e00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 msgr2=0x7faf180d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T12:36:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:58.223+0000 7faf17e00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 msgr2=0x7faf180d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:36:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:58.223+0000 7faf17e00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:36:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:58.223+0000 7faf17e00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-28T12:36:59.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.816+0000 7faf15600640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 msgr2=0x7faf180d56f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:36:59.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.816+0000 7faf15600640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.157:3300/0 conn(0x7faf180b3cb0 0x7faf180d56f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:36:59.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.816+0000 7faf15600640 1 -- 172.21.15.131:0/971078238 >> v1:172.21.15.157:6789/0 conn(0x7faf180d3830 legacy=0x7faf180b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:36:59.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.816+0000 7faf15600640 1 -- 172.21.15.131:0/971078238 >> v1:172.21.15.131:6789/0 conn(0x7faf1815f180 legacy=0x7faf180d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:36:59.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.816+0000 7faf15600640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:36:59.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.816+0000 7faf15600640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:36:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.816+0000 7faf15600640 1 -- 172.21.15.131:0/971078238 --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7faf1815f560 con 0x7faef40061f0 2024-04-28T12:36:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.816+0000 7faf15600640 1 -- 172.21.15.131:0/971078238 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faef4008d60 con 0x7faef4003730 2024-04-28T12:36:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.816+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:36:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.816+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:36:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.817+0000 7faf0ea00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 msgr2=0x7faef40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-28T12:36:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.817+0000 7faf0ea00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 msgr2=0x7faef40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:36:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.817+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:36:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.817+0000 7faf17400640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faef4005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T12:36:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.817+0000 7faf17400640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faef4005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:36:59.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.817+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:36:59.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.817+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:36:59.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:36:59.817+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.018+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.018+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.019+0000 7faf0ea00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 msgr2=0x7faef40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-28T12:37:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.019+0000 7faf0ea00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 msgr2=0x7faef40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.019+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.019+0000 7faf17400640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faef4005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T12:37:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.019+0000 7faf17400640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faef4005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.019+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:00.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.019+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-28T12:37:00.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.019+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-28T12:37:00.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.420+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:00.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.420+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:00.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.420+0000 7faf0ea00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 msgr2=0x7faef40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T12:37:00.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.420+0000 7faf0ea00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 msgr2=0x7faef40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:00.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.420+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:00.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.420+0000 7faf17400640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faef4005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-28T12:37:00.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.420+0000 7faf17400640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faef4005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:00.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.420+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:00.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.421+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-28T12:37:00.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:00.421+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-28T12:37:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:01.222+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:01.222+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:01.223+0000 7faf0ea00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 msgr2=0x7faef40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T12:37:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:01.223+0000 7faf0ea00640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 msgr2=0x7faef40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:01.223+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:01.223+0000 7faf17400640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faef4005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-28T12:37:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:01.223+0000 7faf17400640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faef4005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:01.223+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:01.223+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-28T12:37:01.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:01.223+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-28T12:37:01.995 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:01.998+0000 7f51361ee040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-28T12:37:02.188 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:02.191+0000 7f51361ee040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:37:02.254 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:02.257+0000 7f51361ee040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-28T12:37:02.529 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:02.532+0000 7f51361ee040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-28T12:37:02.582 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:02.585+0000 7f51361ee040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-28T12:37:02.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.815+0000 7faf15600640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faef4005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:37:02.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.815+0000 7faf15600640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:02.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.816+0000 7faf15600640 1 -- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 msgr2=0x7faef40086b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:37:02.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.816+0000 7faf15600640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:02.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.816+0000 7faf17e00640 1 -- 172.21.15.131:0/971078238 reap_dead start 2024-04-28T12:37:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.816+0000 7faf15600640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.816+0000 7faf15600640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.816+0000 7faf15600640 1 -- 172.21.15.131:0/971078238 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf180d09c0 con 0x7faf180d3830 2024-04-28T12:37:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.816+0000 7faf15600640 1 -- 172.21.15.131:0/971078238 --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7faf1815f560 con 0x7faef40061f0 2024-04-28T12:37:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.816+0000 7faf15600640 1 -- 172.21.15.131:0/971078238 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faef4008d60 con 0x7faef4003730 2024-04-28T12:37:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.816+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.816+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> v2:172.21.15.131:3300/0 conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.817+0000 7faf17400640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faef4005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-28T12:37:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.817+0000 7faf17400640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faef4005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.817+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:02.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.817+0000 7faf17400640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:02.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.817+0000 7faf0ea00640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faef4005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:37:02.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.817+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:02.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.817+0000 7faf0ea00640 1 -- 172.21.15.131:0/971078238 >> v1:172.21.15.157:6789/0 conn(0x7faf180d3830 legacy=0x7faef400b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:02.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.817+0000 7faf16a00640 1 -- 172.21.15.131:0/971078238 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2452060604 0 0) 0x7faf180d09c0 con 0x7faf180d3830 2024-04-28T12:37:02.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.817+0000 7faf0ea00640 1 -- 172.21.15.131:0/971078238 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef400db10 con 0x7faef40061f0 2024-04-28T12:37:02.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.821+0000 7faf0ea00640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef40061f0 0x7faef40086b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf00007c30 tx=0x7faf0002a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe251a26735264 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:02.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.822+0000 7faf16a00640 1 -- 172.21.15.131:0/971078238 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0002f680 con 0x7faef40061f0 2024-04-28T12:37:02.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.822+0000 7faf16a00640 1 -- 172.21.15.131:0/971078238 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf00036070 con 0x7faef40061f0 2024-04-28T12:37:02.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.822+0000 7faf16a00640 1 -- 172.21.15.131:0/971078238 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0002fa60 con 0x7faef40061f0 2024-04-28T12:37:02.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.823+0000 7faf1ce00640 1 -- 172.21.15.131:0/971078238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef40061f0 msgr2=0x7faef40086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:02.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.823+0000 7faf1ce00640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef40061f0 0x7faef40086b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf00007c30 tx=0x7faf0002a9c0 comp rx=0 tx=0).stop 2024-04-28T12:37:02.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.824+0000 7faf1ce00640 1 -- 172.21.15.131:0/971078238 shutdown_connections 2024-04-28T12:37:02.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.824+0000 7faf1ce00640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef40061f0 0x7faef40086b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:02.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.824+0000 7faf1ce00640 1 --2- 172.21.15.131:0/971078238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faef4005b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:02.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.824+0000 7faf1ce00640 1 -- 172.21.15.131:0/971078238 >> 172.21.15.131:0/971078238 conn(0x7faf180d4bd0 msgr2=0x7faf180d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:02.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.824+0000 7faf1ce00640 1 -- 172.21.15.131:0/971078238 shutdown_connections 2024-04-28T12:37:02.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.824+0000 7faf1ce00640 1 -- 172.21.15.131:0/971078238 wait complete. 2024-04-28T12:37:02.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.827+0000 7faf1ce00640 1 Processor -- start 2024-04-28T12:37:02.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.827+0000 7faf1ce00640 1 -- start start 2024-04-28T12:37:02.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf1ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faf180b8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:02.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf1ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf180d3830 0x7faf180b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:02.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf1ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf180ba4e0 0x7faf180b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:02.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf1ce00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faf180d11c0 con 0x7faf180d3830 2024-04-28T12:37:02.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf1ce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faf180d2a40 con 0x7faf180ba4e0 2024-04-28T12:37:02.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf1ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faf180b77f0 con 0x7faef4003730 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf17400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faf180b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf17400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faf180b8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:48702/0 (socket says 172.21.15.131:48702) 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf17400640 1 -- 172.21.15.131:0/3508471156 learned_addr learned my addr 172.21.15.131:0/3508471156 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf17e00640 1 --2- 172.21.15.131:0/3508471156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf180ba4e0 0x7faf180b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf17400640 1 -- 172.21.15.131:0/3508471156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faf180b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf17400640 1 -- 172.21.15.131:0/3508471156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faf180b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf17400640 1 --2- 172.21.15.131:0/3508471156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faf180b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.829+0000 7faf17400640 1 --2- 172.21.15.131:0/3508471156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faf180b8800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.830+0000 7faf0ea00640 1 --2- 172.21.15.131:0/3508471156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf180d3830 0x7faf180b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.830+0000 7faf0ea00640 1 -- 172.21.15.131:0/3508471156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 msgr2=0x7faf180b8800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.830+0000 7faf0ea00640 1 --2- 172.21.15.131:0/3508471156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faf180b8800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.830+0000 7faf0ea00640 1 -- 172.21.15.131:0/3508471156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf180ba4e0 msgr2=0x7faf180b91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:02.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.830+0000 7faf0ea00640 1 --2- 172.21.15.131:0/3508471156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf180ba4e0 0x7faf180b91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:02.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.830+0000 7faf0ea00640 1 -- 172.21.15.131:0/3508471156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf180b7a40 con 0x7faf180d3830 2024-04-28T12:37:02.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.831+0000 7faf0ea00640 1 --2- 172.21.15.131:0/3508471156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf180d3830 0x7faf180b9fa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faef40061c0 tx=0x7faf000010c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:02.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.831+0000 7faf0fe00640 1 -- 172.21.15.131:0/3508471156 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf000110b0 con 0x7faf180d3830 2024-04-28T12:37:02.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.831+0000 7faf0fe00640 1 -- 172.21.15.131:0/3508471156 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0003f070 con 0x7faf180d3830 2024-04-28T12:37:02.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.831+0000 7faf1ce00640 1 -- 172.21.15.131:0/3508471156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf180b7d20 con 0x7faf180d3830 2024-04-28T12:37:02.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.831+0000 7faf0fe00640 1 -- 172.21.15.131:0/3508471156 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00031e00 con 0x7faf180d3830 2024-04-28T12:37:02.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.831+0000 7faf1ce00640 1 -- 172.21.15.131:0/3508471156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf180b8d40 con 0x7faf180d3830 2024-04-28T12:37:02.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.832+0000 7faf0fe00640 1 -- 172.21.15.131:0/3508471156 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faf00030070 con 0x7faf180d3830 2024-04-28T12:37:02.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.832+0000 7faf0fe00640 1 -- 172.21.15.131:0/3508471156 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faf00044cd0 con 0x7faf180d3830 2024-04-28T12:37:02.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.833+0000 7faf1ce00640 1 -- 172.21.15.131:0/3508471156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf180506c0 con 0x7faf180d3830 2024-04-28T12:37:02.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.844+0000 7faf0fe00640 1 -- 172.21.15.131:0/3508471156 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7faf00044470 con 0x7faf180d3830 2024-04-28T12:37:02.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.918+0000 7faf1ce00640 1 -- 172.21.15.131:0/3508471156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7faf180b9d80 con 0x7faf180d3830 2024-04-28T12:37:02.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.923+0000 7faf0fe00640 1 -- 172.21.15.131:0/3508471156 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7faf00044650 con 0x7faf180d3830 2024-04-28T12:37:02.919 INFO:teuthology.orchestra.run.smithi131.stderr:adjusted tunables profile to default 2024-04-28T12:37:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.924+0000 7faf1ce00640 1 -- 172.21.15.131:0/3508471156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf180d3830 msgr2=0x7faf180b9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.924+0000 7faf1ce00640 1 --2- 172.21.15.131:0/3508471156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf180d3830 0x7faf180b9fa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faef40061c0 tx=0x7faf000010c0 comp rx=0 tx=0).stop 2024-04-28T12:37:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.924+0000 7faf1ce00640 1 -- 172.21.15.131:0/3508471156 shutdown_connections 2024-04-28T12:37:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.924+0000 7faf1ce00640 1 --2- 172.21.15.131:0/3508471156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf180ba4e0 0x7faf180b91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.924+0000 7faf1ce00640 1 --2- 172.21.15.131:0/3508471156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf180d3830 0x7faf180b9fa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.924+0000 7faf1ce00640 1 --2- 172.21.15.131:0/3508471156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faef4003730 0x7faf180b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.924+0000 7faf1ce00640 1 -- 172.21.15.131:0/3508471156 >> 172.21.15.131:0/3508471156 conn(0x7faf180d4bd0 msgr2=0x7faf18156450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.924+0000 7faf1ce00640 1 -- 172.21.15.131:0/3508471156 shutdown_connections 2024-04-28T12:37:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:02.924+0000 7faf1ce00640 1 -- 172.21.15.131:0/3508471156 wait complete. 2024-04-28T12:37:02.939 INFO:tasks.ceph:check_enable_crimson: False 2024-04-28T12:37:02.940 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-28T12:37:02.940 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:37:02.940 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-28T12:37:02.970 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:02.974+0000 7fb17f1bc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-28T12:37:03.010 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:37:03.010 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-28T12:37:03.127 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:37:03.127 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-28T12:37:03.187 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:03.190+0000 7fb17f1bc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:37:03.238 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:37:03.238 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-28T12:37:03.253 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:03.256+0000 7fb17f1bc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-28T12:37:03.318 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:37:03.318 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-28T12:37:03.399 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:37:03.399 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-28T12:37:03.464 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:37:03.464 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-28T12:37:03.553 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:03.556+0000 7f51361ee040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-28T12:37:03.556 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:03.560+0000 7fb17f1bc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-28T12:37:03.577 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:37:03.577 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-28T12:37:03.615 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:03.618+0000 7fb17f1bc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-28T12:37:03.661 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:03.664+0000 7f51361ee040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-28T12:37:03.686 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 15970349-8188-4409-af6d-1ac6680cddd8 0 2024-04-28T12:37:03.714 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:03.717+0000 7f51361ee040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-28T12:37:03.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.891+0000 7f46f3200640 1 Processor -- start 2024-04-28T12:37:03.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.892+0000 7f46f3200640 1 -- start start 2024-04-28T12:37:03.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.893+0000 7f46f3200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f46f415f180 0x7f46f40d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:03.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f3200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f46f40b3cb0 0x7f46f40d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:03.891 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f40d1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f3200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f46f415f560 con 0x7f46f40b3cb0 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f3200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f46f40d2d70 con 0x7f46f415f180 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f46f40d06e0 con 0x7f46f40d3660 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f2800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f40d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f2800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f40d1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:43802/0 (socket says 172.21.15.157:43802) 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f2800640 1 -- 172.21.15.157:0/916148918 learned_addr learned my addr 172.21.15.157:0/916148918 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f1400640 1 --2- 172.21.15.157:0/916148918 >> v2:172.21.15.131:3300/0 conn(0x7f46f40b3cb0 0x7f46f40d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f2800640 1 -- 172.21.15.157:0/916148918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 msgr2=0x7f46f40d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f2800640 1 -- 172.21.15.157:0/916148918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 msgr2=0x7f46f40d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f2800640 1 --2- 172.21.15.157:0/916148918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f40d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f2800640 1 --2- 172.21.15.157:0/916148918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f40d1570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.894+0000 7f46f1e00640 1 --2- 172.21.15.157:0/916148918 >> v2:172.21.15.157:3300/0 conn(0x7f46f415f180 0x7f46f40d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.895+0000 7f46f1400640 1 -- 172.21.15.157:0/916148918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 msgr2=0x7f46f40d1570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:37:03.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.895+0000 7f46f1400640 1 --2- 172.21.15.157:0/916148918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f40d1570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.895+0000 7f46f1400640 1 -- 172.21.15.157:0/916148918 >> v2:172.21.15.157:3300/0 conn(0x7f46f415f180 msgr2=0x7f46f40d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.895+0000 7f46f1400640 1 --2- 172.21.15.157:0/916148918 >> v2:172.21.15.157:3300/0 conn(0x7f46f415f180 0x7f46f40d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:03.893 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.895+0000 7f46f1400640 1 -- 172.21.15.157:0/916148918 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f40d09c0 con 0x7f46f40b3cb0 2024-04-28T12:37:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.896+0000 7f46f1400640 1 --2- 172.21.15.157:0/916148918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f40b3cb0 0x7f46f40d3120 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f46f408d260 tx=0x7f46e002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8415b69f4abc79fd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.896+0000 7f46f0a00640 1 -- 172.21.15.157:0/916148918 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0037070 con 0x7f46f40b3cb0 2024-04-28T12:37:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.896+0000 7f46f0a00640 1 -- 172.21.15.157:0/916148918 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e0032440 con 0x7f46f40b3cb0 2024-04-28T12:37:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.896+0000 7f46f0a00640 1 -- 172.21.15.157:0/916148918 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e002ae60 con 0x7f46f40b3cb0 2024-04-28T12:37:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.897+0000 7f46f3200640 1 -- 172.21.15.157:0/916148918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f40b3cb0 msgr2=0x7f46f40d3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.897+0000 7f46f3200640 1 --2- 172.21.15.157:0/916148918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f40b3cb0 0x7f46f40d3120 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f46f408d260 tx=0x7f46e002a110 comp rx=0 tx=0).stop 2024-04-28T12:37:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.897+0000 7f46f3200640 1 -- 172.21.15.157:0/916148918 shutdown_connections 2024-04-28T12:37:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.897+0000 7f46f3200640 1 --2- 172.21.15.157:0/916148918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f40b3cb0 0x7f46f40d3120 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.897+0000 7f46f3200640 1 --2- 172.21.15.157:0/916148918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f40d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:03.894 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.897+0000 7f46f3200640 1 --2- 172.21.15.157:0/916148918 >> v2:172.21.15.157:3300/0 conn(0x7f46f415f180 0x7f46f40d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:03.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.897+0000 7f46f3200640 1 -- 172.21.15.157:0/916148918 >> 172.21.15.157:0/916148918 conn(0x7f46f40d4bd0 msgr2=0x7f46f40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:03.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.897+0000 7f46f3200640 1 -- 172.21.15.157:0/916148918 shutdown_connections 2024-04-28T12:37:03.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.897+0000 7f46f3200640 1 -- 172.21.15.157:0/916148918 wait complete. 2024-04-28T12:37:03.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.899+0000 7f46f3200640 1 Processor -- start 2024-04-28T12:37:03.895 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.899+0000 7f46f3200640 1 -- start start 2024-04-28T12:37:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46f40b3cb0 0x7f46f4158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f41587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f415f180 0x7f46f41f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f3200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f46f40d11c0 con 0x7f46f415f180 2024-04-28T12:37:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46f40d2a40 con 0x7f46f40b3cb0 2024-04-28T12:37:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f46f40c8f60 con 0x7f46f40d3660 2024-04-28T12:37:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46f40b3cb0 0x7f46f4158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46f40b3cb0 0x7f46f4158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39430/0 (socket says 172.21.15.157:39430) 2024-04-28T12:37:03.898 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1e00640 1 -- 172.21.15.157:0/1686948418 learned_addr learned my addr 172.21.15.157:0/1686948418 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1400640 1 --2- 172.21.15.157:0/1686948418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f41587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1400640 1 -- 172.21.15.157:0/1686948418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 msgr2=0x7f46f41587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1400640 1 -- 172.21.15.157:0/1686948418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 msgr2=0x7f46f41587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1400640 1 --2- 172.21.15.157:0/1686948418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f41587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1400640 1 --2- 172.21.15.157:0/1686948418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f41587b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1e00640 1 -- 172.21.15.157:0/1686948418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 msgr2=0x7f46f41587b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1e00640 1 --2- 172.21.15.157:0/1686948418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f41587b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1e00640 1 -- 172.21.15.157:0/1686948418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f415f180 msgr2=0x7f46f41f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1e00640 1 --2- 172.21.15.157:0/1686948418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f415f180 0x7f46f41f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.900+0000 7f46f1e00640 1 -- 172.21.15.157:0/1686948418 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f40c9240 con 0x7f46f40b3cb0 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.901+0000 7f46f1e00640 1 --2- 172.21.15.157:0/1686948418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46f40b3cb0 0x7f46f4158270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f46e400e870 tx=0x7f46e400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.901+0000 7f46ea000640 1 -- 172.21.15.157:0/1686948418 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4017070 con 0x7f46f40b3cb0 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.901+0000 7f46ea000640 1 -- 172.21.15.157:0/1686948418 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e400c940 con 0x7f46f40b3cb0 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.901+0000 7f46ea000640 1 -- 172.21.15.157:0/1686948418 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e400cd30 con 0x7f46f40b3cb0 2024-04-28T12:37:03.899 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.901+0000 7f46f3200640 1 -- 172.21.15.157:0/1686948418 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f40c9520 con 0x7f46f40b3cb0 2024-04-28T12:37:03.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.901+0000 7f46f3200640 1 -- 172.21.15.157:0/1686948418 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f40c89a0 con 0x7f46f40b3cb0 2024-04-28T12:37:03.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.901+0000 7f46ea000640 1 -- 172.21.15.157:0/1686948418 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f46e401a070 con 0x7f46f40b3cb0 2024-04-28T12:37:03.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.901+0000 7f46ea000640 1 -- 172.21.15.157:0/1686948418 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f46e401c790 con 0x7f46f40b3cb0 2024-04-28T12:37:03.900 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.902+0000 7f46f3200640 1 -- 172.21.15.157:0/1686948418 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f406e770 con 0x7f46f40b3cb0 2024-04-28T12:37:03.902 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.905+0000 7f46ea000640 1 -- 172.21.15.157:0/1686948418 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f46f406e770 con 0x7f46f40b3cb0 2024-04-28T12:37:03.916 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:03.920+0000 7f51361ee040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-28T12:37:03.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.987+0000 7f46f3200640 1 -- 172.21.15.157:0/1686948418 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "15970349-8188-4409-af6d-1ac6680cddd8", "id": 0} v 0) -- 0x7f46f40b74a0 con 0x7f46f40b3cb0 2024-04-28T12:37:03.985 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:03.988+0000 7f51361ee040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-28T12:37:03.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.994+0000 7f46ea000640 1 -- 172.21.15.157:0/1686948418 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "15970349-8188-4409-af6d-1ac6680cddd8", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f46f40b74a0 con 0x7f46f40b3cb0 2024-04-28T12:37:03.991 INFO:teuthology.orchestra.run.smithi157.stdout:0 2024-04-28T12:37:03.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.995+0000 7f46f3200640 1 -- 172.21.15.157:0/1686948418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46f40b3cb0 msgr2=0x7f46f4158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:03.992 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.995+0000 7f46f3200640 1 --2- 172.21.15.157:0/1686948418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46f40b3cb0 0x7f46f4158270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f46e400e870 tx=0x7f46e400ed30 comp rx=0 tx=0).stop 2024-04-28T12:37:03.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.995+0000 7f46f3200640 1 -- 172.21.15.157:0/1686948418 shutdown_connections 2024-04-28T12:37:03.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.995+0000 7f46f3200640 1 --2- 172.21.15.157:0/1686948418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46f415f180 0x7f46f41f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:03.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.995+0000 7f46f3200640 1 --2- 172.21.15.157:0/1686948418 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46f40d3660 0x7f46f41587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:03.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.995+0000 7f46f3200640 1 --2- 172.21.15.157:0/1686948418 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46f40b3cb0 0x7f46f4158270 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:03.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.995+0000 7f46f3200640 1 -- 172.21.15.157:0/1686948418 >> 172.21.15.157:0/1686948418 conn(0x7f46f40d4bd0 msgr2=0x7f46f415dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:03.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.996+0000 7f46f3200640 1 -- 172.21.15.157:0/1686948418 shutdown_connections 2024-04-28T12:37:03.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:03.996+0000 7f46f3200640 1 -- 172.21.15.157:0/1686948418 wait complete. 2024-04-28T12:37:04.009 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 9ece051d-6eca-401a-b213-c8a72199d9d1 1 2024-04-28T12:37:04.051 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:04.055+0000 7f51361ee040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-28T12:37:04.185 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:04.188+0000 7f51361ee040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:37:04.192 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.195+0000 7fb27ca00640 1 Processor -- start 2024-04-28T12:37:04.192 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.195+0000 7fb27ca00640 1 -- start start 2024-04-28T12:37:04.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb27ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2780b17f0 0x7fb278075730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb27ca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb2780b1bd0 0x7fb2780ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb27ca00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2780cd7e0 con 0x7fb2780b3810 2024-04-28T12:37:04.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb27ca00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb2780c4ab0 con 0x7fb2780b1bd0 2024-04-28T12:37:04.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb27ca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb278075e30 con 0x7fb2780b17f0 2024-04-28T12:37:04.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb277400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2780b17f0 0x7fb278075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.196 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb277e00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb2780b3810 0x7fb2780cd0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53490/0 (socket says 172.21.15.157:53490) 2024-04-28T12:37:04.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb277e00640 1 -- 172.21.15.157:0/1052910260 learned_addr learned my addr 172.21.15.157:0/1052910260 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-28T12:37:04.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb26fe00640 1 --2- 172.21.15.157:0/1052910260 >> v2:172.21.15.131:3300/0 conn(0x7fb2780b1bd0 0x7fb2780ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb277400640 1 -- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2780b17f0 msgr2=0x7fb278075730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-28T12:37:04.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb277400640 1 -- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2780b17f0 msgr2=0x7fb278075730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:04.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb277400640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2780b17f0 0x7fb278075730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:04.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.197+0000 7fb277400640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2780b17f0 0x7fb278075730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:04.197 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3977782515 0 0) 0x7fb2780cd7e0 con 0x7fb2780b3810 2024-04-28T12:37:04.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb25c003520 con 0x7fb2780b3810 2024-04-28T12:37:04.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3805715103 0 0) 0x7fb25c003520 con 0x7fb2780b3810 2024-04-28T12:37:04.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2780b17f0 msgr2=0x7fb278075730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:37:04.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2780b17f0 0x7fb278075730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 >> v2:172.21.15.131:3300/0 conn(0x7fb2780b1bd0 msgr2=0x7fb2780ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:04.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 --2- 172.21.15.157:0/1052910260 >> v2:172.21.15.131:3300/0 conn(0x7fb2780b1bd0 0x7fb2780ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2780cdaa0 con 0x7fb2780b3810 2024-04-28T12:37:04.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fb2700019f0 con 0x7fb2780b3810 2024-04-28T12:37:04.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 >> v1:172.21.15.157:6789/0 conn(0x7fb2780b3810 legacy=0x7fb2780cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:04.198 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb25c003a20 0x7fb25c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2780b17f0 0x7fb25c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c004d40 0x7fb25c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb2780c4ab0 con 0x7fb25c003a20 2024-04-28T12:37:04.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb278075e30 con 0x7fb25c004d40 2024-04-28T12:37:04.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb25c00ba40 con 0x7fb2780b17f0 2024-04-28T12:37:04.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb26fe00640 1 --2- 172.21.15.157:0/1052910260 >> v2:172.21.15.131:3300/0 conn(0x7fb2780b1bd0 0x7fb2780ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:37:04.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.198+0000 7fb277e00640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c004d40 0x7fb25c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.199+0000 7fb277400640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb25c003a20 0x7fb25c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.199+0000 7fb26fe00640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2780b17f0 0x7fb25c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.199+0000 7fb277e00640 1 -- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2780b17f0 msgr2=0x7fb25c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:04.199 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.199+0000 7fb277e00640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2780b17f0 0x7fb25c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.199+0000 7fb277e00640 1 -- 172.21.15.157:0/1052910260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb25c003a20 msgr2=0x7fb25c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:04.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.199+0000 7fb277e00640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb25c003a20 0x7fb25c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.200 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.199+0000 7fb277e00640 1 -- 172.21.15.157:0/1052910260 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25c003520 con 0x7fb25c004d40 2024-04-28T12:37:04.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.200+0000 7fb277e00640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c004d40 0x7fb25c0051c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb27000a8a0 tx=0x7fb25c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee4f983d9e17d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:04.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.201+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27000a4c0 con 0x7fb25c004d40 2024-04-28T12:37:04.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.201+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27002f070 con 0x7fb25c004d40 2024-04-28T12:37:04.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.201+0000 7fb276a00640 1 -- 172.21.15.157:0/1052910260 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27002bd30 con 0x7fb25c004d40 2024-04-28T12:37:04.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.201+0000 7fb27ca00640 1 -- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c004d40 msgr2=0x7fb25c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:04.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.201+0000 7fb27ca00640 1 --2- 172.21.15.157:0/1052910260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb25c004d40 0x7fb25c0051c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb27000a8a0 tx=0x7fb25c005830 comp rx=0 tx=0).stop 2024-04-28T12:37:04.201 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.201+0000 7fb277e00640 1 -- 172.21.15.157:0/1052910260 reap_dead start 2024-04-28T12:37:04.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.201+0000 7fb27ca00640 1 -- 172.21.15.157:0/1052910260 shutdown_connections 2024-04-28T12:37:04.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.201+0000 7fb27ca00640 1 -- 172.21.15.157:0/1052910260 >> 172.21.15.157:0/1052910260 conn(0x7fb2780b2cf0 msgr2=0x7fb2780b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:04.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.201+0000 7fb27ca00640 1 -- 172.21.15.157:0/1052910260 shutdown_connections 2024-04-28T12:37:04.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.201+0000 7fb27ca00640 1 -- 172.21.15.157:0/1052910260 wait complete. 2024-04-28T12:37:04.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.204+0000 7fb27ca00640 1 Processor -- start 2024-04-28T12:37:04.202 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.204+0000 7fb27ca00640 1 -- start start 2024-04-28T12:37:04.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb27ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c004d40 0x7fb2781f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb27ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2781564b0 0x7fb2780af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.204 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb27ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb27809aa40 0x7fb2780afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb27ca00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb278075730 con 0x7fb2781564b0 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb27ca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2780ce060 con 0x7fb27809aa40 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb27ca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2780758b0 con 0x7fb25c004d40 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb277400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c004d40 0x7fb2781f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb277400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c004d40 0x7fb2781f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:43834/0 (socket says 172.21.15.157:43834) 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb277400640 1 -- 172.21.15.157:0/4210182390 learned_addr learned my addr 172.21.15.157:0/4210182390 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb277e00640 1 --2- 172.21.15.157:0/4210182390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb27809aa40 0x7fb2780afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb26fe00640 1 --2- 172.21.15.157:0/4210182390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2781564b0 0x7fb2780af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb277400640 1 -- 172.21.15.157:0/4210182390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c004d40 msgr2=0x7fb2781f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb277400640 1 -- 172.21.15.157:0/4210182390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c004d40 msgr2=0x7fb2781f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb277400640 1 --2- 172.21.15.157:0/4210182390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c004d40 0x7fb2781f2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.206+0000 7fb277400640 1 --2- 172.21.15.157:0/4210182390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c004d40 0x7fb2781f2b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.207+0000 7fb277e00640 1 -- 172.21.15.157:0/4210182390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c004d40 msgr2=0x7fb2781f2b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:37:04.205 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.207+0000 7fb277e00640 1 --2- 172.21.15.157:0/4210182390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c004d40 0x7fb2781f2b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.207+0000 7fb277e00640 1 -- 172.21.15.157:0/4210182390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2781564b0 msgr2=0x7fb2780af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:04.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.207+0000 7fb277e00640 1 --2- 172.21.15.157:0/4210182390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2781564b0 0x7fb2780af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.207+0000 7fb277e00640 1 -- 172.21.15.157:0/4210182390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2781574b0 con 0x7fb27809aa40 2024-04-28T12:37:04.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.207+0000 7fb277e00640 1 --2- 172.21.15.157:0/4210182390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb27809aa40 0x7fb2780afc40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00ba10 tx=0x7fb27815d790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:04.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.207+0000 7fb26f400640 1 -- 172.21.15.157:0/4210182390 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27000a4c0 con 0x7fb27809aa40 2024-04-28T12:37:04.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.207+0000 7fb26f400640 1 -- 172.21.15.157:0/4210182390 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb270004d60 con 0x7fb27809aa40 2024-04-28T12:37:04.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.207+0000 7fb26f400640 1 -- 172.21.15.157:0/4210182390 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270034670 con 0x7fb27809aa40 2024-04-28T12:37:04.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.207+0000 7fb27ca00640 1 -- 172.21.15.157:0/4210182390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb278156ef0 con 0x7fb27809aa40 2024-04-28T12:37:04.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.207+0000 7fb27ca00640 1 -- 172.21.15.157:0/4210182390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2780c7980 con 0x7fb27809aa40 2024-04-28T12:37:04.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.208+0000 7fb26f400640 1 -- 172.21.15.157:0/4210182390 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb270046060 con 0x7fb27809aa40 2024-04-28T12:37:04.206 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.208+0000 7fb26f400640 1 -- 172.21.15.157:0/4210182390 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fb270048800 con 0x7fb27809aa40 2024-04-28T12:37:04.207 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.208+0000 7fb27ca00640 1 -- 172.21.15.157:0/4210182390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2780c5640 con 0x7fb27809aa40 2024-04-28T12:37:04.209 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.212+0000 7fb26f400640 1 -- 172.21.15.157:0/4210182390 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb2781574b0 con 0x7fb27809aa40 2024-04-28T12:37:04.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.291+0000 7fb27ca00640 1 -- 172.21.15.157:0/4210182390 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9ece051d-6eca-401a-b213-c8a72199d9d1", "id": 1} v 0) -- 0x7fb2780506c0 con 0x7fb27809aa40 2024-04-28T12:37:04.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.298+0000 7fb26f400640 1 -- 172.21.15.157:0/4210182390 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ece051d-6eca-401a-b213-c8a72199d9d1", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fb27002ce20 con 0x7fb27809aa40 2024-04-28T12:37:04.295 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-28T12:37:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.299+0000 7fb27ca00640 1 -- 172.21.15.157:0/4210182390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb27809aa40 msgr2=0x7fb2780afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.299+0000 7fb27ca00640 1 --2- 172.21.15.157:0/4210182390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb27809aa40 0x7fb2780afc40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00ba10 tx=0x7fb27815d790 comp rx=0 tx=0).stop 2024-04-28T12:37:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.299+0000 7fb27ca00640 1 -- 172.21.15.157:0/4210182390 shutdown_connections 2024-04-28T12:37:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.299+0000 7fb27ca00640 1 --2- 172.21.15.157:0/4210182390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb27809aa40 0x7fb2780afc40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.299+0000 7fb27ca00640 1 --2- 172.21.15.157:0/4210182390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2781564b0 0x7fb2780af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.299+0000 7fb27ca00640 1 --2- 172.21.15.157:0/4210182390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb25c004d40 0x7fb2781f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.299+0000 7fb27ca00640 1 -- 172.21.15.157:0/4210182390 >> 172.21.15.157:0/4210182390 conn(0x7fb2780b2cf0 msgr2=0x7fb2780b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:04.296 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.299+0000 7fb27ca00640 1 -- 172.21.15.157:0/4210182390 shutdown_connections 2024-04-28T12:37:04.297 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.299+0000 7fb27ca00640 1 -- 172.21.15.157:0/4210182390 wait complete. 2024-04-28T12:37:04.313 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new e1f72f87-0adf-410a-aa9c-d03196484282 2 2024-04-28T12:37:04.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.540+0000 7f6a93000640 1 Processor -- start 2024-04-28T12:37:04.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.541+0000 7f6a93000640 1 -- start start 2024-04-28T12:37:04.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.542+0000 7f6a93000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 0x7f6a8c0d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.542+0000 7f6a93000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a8c0d1120 con 0x7f6a8c0d35c0 2024-04-28T12:37:04.544 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.542+0000 7f6a93000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a8c0b34f0 con 0x7f6a8c0aff80 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.542+0000 7f6a93000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6a8c0b37d0 con 0x7f6a8c1604d0 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.542+0000 7f6a91c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 0x7f6a8c0d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a91c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 0x7f6a8c0d15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:43850/0 (socket says 172.21.15.157:43850) 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a91c00640 1 -- 172.21.15.157:0/2367560646 learned_addr learned my addr 172.21.15.157:0/2367560646 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a92600640 1 --1- 172.21.15.157:0/2367560646 >> v1:172.21.15.157:6789/0 conn(0x7f6a8c0aff80 0x7f6a8c0b1260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53498/0 (socket says 172.21.15.157:53498) 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a91c00640 1 -- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 msgr2=0x7f6a8c0d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a91c00640 1 -- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 msgr2=0x7f6a8c0d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a91c00640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 0x7f6a8c0d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a91c00640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 0x7f6a8c0d15d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 573869072 0 0) 0x7f6a8c0b34f0 con 0x7f6a8c0aff80 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a74003520 con 0x7f6a8c0aff80 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2307802096 0 0) 0x7f6a74003520 con 0x7f6a8c0aff80 2024-04-28T12:37:04.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 msgr2=0x7f6a8c0d15d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a8be00640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 0x7f6a8c0d15d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 >> v1:172.21.15.131:6789/0 conn(0x7f6a8c0d35c0 legacy=0x7f6a8c0af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c0b3ab0 con 0x7f6a8c0aff80 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f6a840019f0 con 0x7f6a8c0aff80 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 >> v1:172.21.15.157:6789/0 conn(0x7f6a8c0aff80 legacy=0x7f6a8c0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.543+0000 7f6a8be00640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 0x7f6a74003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a8be00640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a74004050 0x7f6a74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a8be00640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a74004d40 0x7f6a740051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c0b37d0 con 0x7f6a74004d40 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7400bbb0 con 0x7f6a74004050 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6a7400bdb0 con 0x7f6a8c1604d0 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a91c00640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 0x7f6a74003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a91200640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a74004050 0x7f6a74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a92600640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a74004d40 0x7f6a740051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a91c00640 1 -- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a74004050 msgr2=0x7f6a74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a91c00640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a74004050 0x7f6a74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a91c00640 1 -- 172.21.15.157:0/2367560646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a74004d40 msgr2=0x7f6a740051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a91c00640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a74004d40 0x7f6a740051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.544+0000 7f6a91c00640 1 -- 172.21.15.157:0/2367560646 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a74003520 con 0x7f6a8c1604d0 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.545+0000 7f6a91c00640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 0x7f6a74003a20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009890 tx=0x7f6a74005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c49c196ab640262 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.546+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80037070 con 0x7f6a8c1604d0 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.546+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a8002ad00 con 0x7f6a8c1604d0 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.546+0000 7f6a8be00640 1 -- 172.21.15.157:0/2367560646 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8000f040 con 0x7f6a8c1604d0 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.546+0000 7f6a93000640 1 -- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 msgr2=0x7f6a74003a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.546+0000 7f6a93000640 1 --2- 172.21.15.157:0/2367560646 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1604d0 0x7f6a74003a20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009890 tx=0x7f6a74005830 comp rx=0 tx=0).stop 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.546+0000 7f6a92600640 1 -- 172.21.15.157:0/2367560646 reap_dead start 2024-04-28T12:37:04.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.546+0000 7f6a93000640 1 -- 172.21.15.157:0/2367560646 shutdown_connections 2024-04-28T12:37:04.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.546+0000 7f6a93000640 1 -- 172.21.15.157:0/2367560646 >> 172.21.15.157:0/2367560646 conn(0x7f6a8c0d2aa0 msgr2=0x7f6a8c0d2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:04.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.546+0000 7f6a93000640 1 -- 172.21.15.157:0/2367560646 shutdown_connections 2024-04-28T12:37:04.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.546+0000 7f6a93000640 1 -- 172.21.15.157:0/2367560646 wait complete. 2024-04-28T12:37:04.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.548+0000 7f6a93000640 1 Processor -- start 2024-04-28T12:37:04.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.548+0000 7f6a93000640 1 -- start start 2024-04-28T12:37:04.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.549+0000 7f6a93000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a8c1604d0 0x7f6a8c1f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.549+0000 7f6a93000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a8c156220 0x7f6a8c0b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.549+0000 7f6a93000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1f50a0 0x7f6a8c0b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.550 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.549+0000 7f6a93000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c0b3d90 con 0x7f6a8c156220 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.549+0000 7f6a93000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c1608b0 con 0x7f6a8c1604d0 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.549+0000 7f6a93000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6a8c0b3f10 con 0x7f6a8c1f50a0 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.549+0000 7f6a91c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a8c1604d0 0x7f6a8c1f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.549+0000 7f6a91c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a8c1604d0 0x7f6a8c1f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39466/0 (socket says 172.21.15.157:39466) 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.549+0000 7f6a91c00640 1 -- 172.21.15.157:0/485117346 learned_addr learned my addr 172.21.15.157:0/485117346 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.549+0000 7f6a92600640 1 --2- 172.21.15.157:0/485117346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1f50a0 0x7f6a8c0b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a91200640 1 --2- 172.21.15.157:0/485117346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a8c156220 0x7f6a8c0b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a92600640 1 -- 172.21.15.157:0/485117346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1f50a0 msgr2=0x7f6a8c0b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a92600640 1 -- 172.21.15.157:0/485117346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1f50a0 msgr2=0x7f6a8c0b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a92600640 1 --2- 172.21.15.157:0/485117346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1f50a0 0x7f6a8c0b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a92600640 1 --2- 172.21.15.157:0/485117346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1f50a0 0x7f6a8c0b19a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a91c00640 1 -- 172.21.15.157:0/485117346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1f50a0 msgr2=0x7f6a8c0b19a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a91c00640 1 --2- 172.21.15.157:0/485117346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1f50a0 0x7f6a8c0b19a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a91c00640 1 -- 172.21.15.157:0/485117346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a8c156220 msgr2=0x7f6a8c0b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:04.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a91c00640 1 --2- 172.21.15.157:0/485117346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a8c156220 0x7f6a8c0b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a91c00640 1 -- 172.21.15.157:0/485117346 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c1f5600 con 0x7f6a8c1604d0 2024-04-28T12:37:04.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a91c00640 1 --2- 172.21.15.157:0/485117346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a8c1604d0 0x7f6a8c1f4b60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6a800099b0 tx=0x7f6a80004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:04.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a89600640 1 -- 172.21.15.157:0/485117346 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80042020 con 0x7f6a8c1604d0 2024-04-28T12:37:04.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a89600640 1 -- 172.21.15.157:0/485117346 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a8000f000 con 0x7f6a8c1604d0 2024-04-28T12:37:04.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.550+0000 7f6a89600640 1 -- 172.21.15.157:0/485117346 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80037040 con 0x7f6a8c1604d0 2024-04-28T12:37:04.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.551+0000 7f6a93000640 1 -- 172.21.15.157:0/485117346 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8c1f58e0 con 0x7f6a8c1604d0 2024-04-28T12:37:04.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.551+0000 7f6a93000640 1 -- 172.21.15.157:0/485117346 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a8c0c7d90 con 0x7f6a8c1604d0 2024-04-28T12:37:04.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.551+0000 7f6a89600640 1 -- 172.21.15.157:0/485117346 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6a80045020 con 0x7f6a8c1604d0 2024-04-28T12:37:04.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.551+0000 7f6a89600640 1 -- 172.21.15.157:0/485117346 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f6a8003e7e0 con 0x7f6a8c1604d0 2024-04-28T12:37:04.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.552+0000 7f6a93000640 1 -- 172.21.15.157:0/485117346 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a8c074060 con 0x7f6a8c1604d0 2024-04-28T12:37:04.552 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.554+0000 7f6a89600640 1 -- 172.21.15.157:0/485117346 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6a8c074060 con 0x7f6a8c1604d0 2024-04-28T12:37:04.629 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:04.632+0000 7fb17f1bc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-28T12:37:04.629 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.633+0000 7f6a93000640 1 -- 172.21.15.157:0/485117346 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e1f72f87-0adf-410a-aa9c-d03196484282", "id": 2} v 0) -- 0x7f6a8c0506c0 con 0x7f6a8c1604d0 2024-04-28T12:37:04.637 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.640+0000 7f6a89600640 1 -- 172.21.15.157:0/485117346 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e1f72f87-0adf-410a-aa9c-d03196484282", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f6a8004c030 con 0x7f6a8c1604d0 2024-04-28T12:37:04.637 INFO:teuthology.orchestra.run.smithi157.stdout:2 2024-04-28T12:37:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.641+0000 7f6a93000640 1 -- 172.21.15.157:0/485117346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a8c1604d0 msgr2=0x7f6a8c1f4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.641+0000 7f6a93000640 1 --2- 172.21.15.157:0/485117346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a8c1604d0 0x7f6a8c1f4b60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6a800099b0 tx=0x7f6a80004730 comp rx=0 tx=0).stop 2024-04-28T12:37:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.642+0000 7f6a93000640 1 -- 172.21.15.157:0/485117346 shutdown_connections 2024-04-28T12:37:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.642+0000 7f6a93000640 1 --2- 172.21.15.157:0/485117346 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a8c1f50a0 0x7f6a8c0b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.642+0000 7f6a93000640 1 --2- 172.21.15.157:0/485117346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a8c156220 0x7f6a8c0b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.642+0000 7f6a93000640 1 --2- 172.21.15.157:0/485117346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a8c1604d0 0x7f6a8c1f4b60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.642+0000 7f6a93000640 1 -- 172.21.15.157:0/485117346 >> 172.21.15.157:0/485117346 conn(0x7f6a8c0d2aa0 msgr2=0x7f6a8c06ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.642+0000 7f6a93000640 1 -- 172.21.15.157:0/485117346 shutdown_connections 2024-04-28T12:37:04.639 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.642+0000 7f6a93000640 1 -- 172.21.15.157:0/485117346 wait complete. 2024-04-28T12:37:04.655 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 7097fba1-6608-41a1-833b-582c898e3cfb 3 2024-04-28T12:37:04.672 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:04.675+0000 7f51361ee040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-28T12:37:04.733 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:04.736+0000 7fb17f1bc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-28T12:37:04.783 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:04.786+0000 7fb17f1bc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-28T12:37:04.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.838+0000 7fcff4a00640 1 Processor -- start 2024-04-28T12:37:04.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.838+0000 7fcff4a00640 1 -- start start 2024-04-28T12:37:04.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.839+0000 7fcff4a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcff015f180 0x7fcff00d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.839+0000 7fcff4a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcff00b3cb0 0x7fcff00d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.839+0000 7fcff4a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff00d3660 0x7fcff00d1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.839+0000 7fcff4a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcff015f560 con 0x7fcff015f180 2024-04-28T12:37:04.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.839+0000 7fcff4a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcff00d2d70 con 0x7fcff00b3cb0 2024-04-28T12:37:04.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.839+0000 7fcff4a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcff00d06e0 con 0x7fcff00d3660 2024-04-28T12:37:04.837 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfefe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff00d3660 0x7fcff00d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfeea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcff00b3cb0 0x7fcff00d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfefe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff00d3660 0x7fcff00d1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:43886/0 (socket says 172.21.15.157:43886) 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfeea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcff00b3cb0 0x7fcff00d3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39478/0 (socket says 172.21.15.157:39478) 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfefe00640 1 -- 172.21.15.157:0/2262350843 learned_addr learned my addr 172.21.15.157:0/2262350843 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfefe00640 1 -- 172.21.15.157:0/2262350843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff00d3660 msgr2=0x7fcff00d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfefe00640 1 -- 172.21.15.157:0/2262350843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff00d3660 msgr2=0x7fcff00d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfefe00640 1 --2- 172.21.15.157:0/2262350843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff00d3660 0x7fcff00d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfef400640 1 --2- 172.21.15.157:0/2262350843 >> v2:172.21.15.131:3300/0 conn(0x7fcff015f180 0x7fcff00d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfefe00640 1 --2- 172.21.15.157:0/2262350843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff00d3660 0x7fcff00d1570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfeea00640 1 -- 172.21.15.157:0/2262350843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff00d3660 msgr2=0x7fcff00d1570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfeea00640 1 --2- 172.21.15.157:0/2262350843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff00d3660 0x7fcff00d1570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfeea00640 1 -- 172.21.15.157:0/2262350843 >> v2:172.21.15.131:3300/0 conn(0x7fcff015f180 msgr2=0x7fcff00d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:04.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfeea00640 1 --2- 172.21.15.157:0/2262350843 >> v2:172.21.15.131:3300/0 conn(0x7fcff015f180 0x7fcff00d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfeea00640 1 -- 172.21.15.157:0/2262350843 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff00d09c0 con 0x7fcff00b3cb0 2024-04-28T12:37:04.839 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.840+0000 7fcfef400640 1 --2- 172.21.15.157:0/2262350843 >> v2:172.21.15.131:3300/0 conn(0x7fcff015f180 0x7fcff00d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:37:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.841+0000 7fcfeea00640 1 --2- 172.21.15.157:0/2262350843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff00b3cb0 0x7fcff00d3120 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcff00b77c0 tx=0x7fcfdc02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849b559751b55636 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.842+0000 7fcfee000640 1 -- 172.21.15.157:0/2262350843 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc037070 con 0x7fcff00b3cb0 2024-04-28T12:37:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.842+0000 7fcfee000640 1 -- 172.21.15.157:0/2262350843 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfdc032440 con 0x7fcff00b3cb0 2024-04-28T12:37:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.842+0000 7fcfee000640 1 -- 172.21.15.157:0/2262350843 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc02ae60 con 0x7fcff00b3cb0 2024-04-28T12:37:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.842+0000 7fcff4a00640 1 -- 172.21.15.157:0/2262350843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff00b3cb0 msgr2=0x7fcff00d3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.842+0000 7fcff4a00640 1 --2- 172.21.15.157:0/2262350843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff00b3cb0 0x7fcff00d3120 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcff00b77c0 tx=0x7fcfdc02a110 comp rx=0 tx=0).stop 2024-04-28T12:37:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.842+0000 7fcff4a00640 1 -- 172.21.15.157:0/2262350843 shutdown_connections 2024-04-28T12:37:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.842+0000 7fcff4a00640 1 --2- 172.21.15.157:0/2262350843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff00d3660 0x7fcff00d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.842+0000 7fcff4a00640 1 --2- 172.21.15.157:0/2262350843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff00b3cb0 0x7fcff00d3120 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.842+0000 7fcff4a00640 1 --2- 172.21.15.157:0/2262350843 >> v2:172.21.15.131:3300/0 conn(0x7fcff015f180 0x7fcff00d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.842+0000 7fcff4a00640 1 -- 172.21.15.157:0/2262350843 >> 172.21.15.157:0/2262350843 conn(0x7fcff00d4bd0 msgr2=0x7fcff00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:04.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.843+0000 7fcff4a00640 1 -- 172.21.15.157:0/2262350843 shutdown_connections 2024-04-28T12:37:04.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.843+0000 7fcff4a00640 1 -- 172.21.15.157:0/2262350843 wait complete. 2024-04-28T12:37:04.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.844+0000 7fcff4a00640 1 Processor -- start 2024-04-28T12:37:04.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.844+0000 7fcff4a00640 1 -- start start 2024-04-28T12:37:04.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.845+0000 7fcff4a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff00b3cb0 0x7fcff00ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.844 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.845+0000 7fcff4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff00d3660 0x7fcff01f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.845+0000 7fcff4a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff015f180 0x7fcff0154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.845+0000 7fcff4a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcff00d11c0 con 0x7fcff00d3660 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.845+0000 7fcff4a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcff00d2a40 con 0x7fcff00b3cb0 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.845+0000 7fcff4a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcff00ceb00 con 0x7fcff015f180 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfef400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff00b3cb0 0x7fcff00ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfef400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff00b3cb0 0x7fcff00ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39492/0 (socket says 172.21.15.157:39492) 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfef400640 1 -- 172.21.15.157:0/760172238 learned_addr learned my addr 172.21.15.157:0/760172238 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfefe00640 1 --2- 172.21.15.157:0/760172238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff015f180 0x7fcff0154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfefe00640 1 -- 172.21.15.157:0/760172238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff015f180 msgr2=0x7fcff0154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfefe00640 1 -- 172.21.15.157:0/760172238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff015f180 msgr2=0x7fcff0154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfefe00640 1 --2- 172.21.15.157:0/760172238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff015f180 0x7fcff0154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfefe00640 1 --2- 172.21.15.157:0/760172238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff015f180 0x7fcff0154fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:04.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfeea00640 1 --2- 172.21.15.157:0/760172238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff00d3660 0x7fcff01f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfef400640 1 -- 172.21.15.157:0/760172238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff015f180 msgr2=0x7fcff0154fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfef400640 1 --2- 172.21.15.157:0/760172238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff015f180 0x7fcff0154fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfef400640 1 -- 172.21.15.157:0/760172238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff00d3660 msgr2=0x7fcff01f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfef400640 1 --2- 172.21.15.157:0/760172238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff00d3660 0x7fcff01f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfef400640 1 -- 172.21.15.157:0/760172238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff01f8b90 con 0x7fcff00b3cb0 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.846+0000 7fcfef400640 1 --2- 172.21.15.157:0/760172238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff00b3cb0 0x7fcff00ce5c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000d800 tx=0x7fcfe000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.847+0000 7fcfe7400640 1 -- 172.21.15.157:0/760172238 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0004030 con 0x7fcff00b3cb0 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.847+0000 7fcfe7400640 1 -- 172.21.15.157:0/760172238 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe00041d0 con 0x7fcff00b3cb0 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.847+0000 7fcfe7400640 1 -- 172.21.15.157:0/760172238 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00096a0 con 0x7fcff00b3cb0 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.847+0000 7fcff4a00640 1 -- 172.21.15.157:0/760172238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff00cdae0 con 0x7fcff00b3cb0 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.847+0000 7fcff4a00640 1 -- 172.21.15.157:0/760172238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff00ce0a0 con 0x7fcff00b3cb0 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.847+0000 7fcfe7400640 1 -- 172.21.15.157:0/760172238 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcfe0009840 con 0x7fcff00b3cb0 2024-04-28T12:37:04.846 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.847+0000 7fcfe7400640 1 -- 172.21.15.157:0/760172238 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fcfe0003610 con 0x7fcff00b3cb0 2024-04-28T12:37:04.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.848+0000 7fcff4a00640 1 -- 172.21.15.157:0/760172238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff00506c0 con 0x7fcff00b3cb0 2024-04-28T12:37:04.847 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.850+0000 7fcfe7400640 1 -- 172.21.15.157:0/760172238 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fcfe0003810 con 0x7fcff00b3cb0 2024-04-28T12:37:04.926 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.929+0000 7fcff4a00640 1 -- 172.21.15.157:0/760172238 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7097fba1-6608-41a1-833b-582c898e3cfb", "id": 3} v 0) -- 0x7fcff015d920 con 0x7fcff00b3cb0 2024-04-28T12:37:04.939 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.942+0000 7fcfe7400640 1 -- 172.21.15.157:0/760172238 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7097fba1-6608-41a1-833b-582c898e3cfb", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fcfe0009b30 con 0x7fcff00b3cb0 2024-04-28T12:37:04.939 INFO:teuthology.orchestra.run.smithi157.stdout:3 2024-04-28T12:37:04.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.943+0000 7fcff4a00640 1 -- 172.21.15.157:0/760172238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff00b3cb0 msgr2=0x7fcff00ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:04.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.943+0000 7fcff4a00640 1 --2- 172.21.15.157:0/760172238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff00b3cb0 0x7fcff00ce5c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000d800 tx=0x7fcfe000dcc0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.943+0000 7fcff4a00640 1 -- 172.21.15.157:0/760172238 shutdown_connections 2024-04-28T12:37:04.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.943+0000 7fcff4a00640 1 --2- 172.21.15.157:0/760172238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff015f180 0x7fcff0154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.943+0000 7fcff4a00640 1 --2- 172.21.15.157:0/760172238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff00d3660 0x7fcff01f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.943+0000 7fcff4a00640 1 --2- 172.21.15.157:0/760172238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff00b3cb0 0x7fcff00ce5c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:04.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.943+0000 7fcff4a00640 1 -- 172.21.15.157:0/760172238 >> 172.21.15.157:0/760172238 conn(0x7fcff00d4bd0 msgr2=0x7fcff00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:04.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.944+0000 7fcff4a00640 1 -- 172.21.15.157:0/760172238 shutdown_connections 2024-04-28T12:37:04.941 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:04.944+0000 7fcff4a00640 1 -- 172.21.15.157:0/760172238 wait complete. 2024-04-28T12:37:04.956 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6 4 2024-04-28T12:37:04.976 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:04.979+0000 7fb17f1bc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-28T12:37:04.982 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:04.986+0000 7f51361ee040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-28T12:37:05.046 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:05.049+0000 7fb17f1bc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-28T12:37:05.047 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:05.050+0000 7f51361ee040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-28T12:37:05.107 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:05.110+0000 7fb17f1bc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-28T12:37:05.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.165+0000 7f4d36000640 1 Processor -- start 2024-04-28T12:37:05.162 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.165+0000 7f4d36000640 1 -- start start 2024-04-28T12:37:05.164 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.167+0000 7f4d36000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4d300b3810 0x7f4d30075730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.164 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.167+0000 7f4d36000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d30075dc0 con 0x7f4d3015f1b0 2024-04-28T12:37:05.164 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.167+0000 7f4d36000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d300ae070 con 0x7f4d300ce240 2024-04-28T12:37:05.164 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.167+0000 7f4d36000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4d3015f590 con 0x7f4d300b3810 2024-04-28T12:37:05.164 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.167+0000 7f4d2fe00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f4d300ce240 0x7f4d300aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53500/0 (socket says 172.21.15.157:53500) 2024-04-28T12:37:05.164 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.167+0000 7f4d2fe00640 1 -- 172.21.15.157:0/1419430131 learned_addr learned my addr 172.21.15.157:0/1419430131 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-28T12:37:05.166 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.167+0000 7f4d35600640 1 --2- 172.21.15.157:0/1419430131 >> v2:172.21.15.131:3300/0 conn(0x7f4d300b3810 0x7f4d30075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.166 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1590835577 0 0) 0x7f4d300ae070 con 0x7f4d300ce240 2024-04-28T12:37:05.166 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d18003520 con 0x7f4d300ce240 2024-04-28T12:37:05.166 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1357146366 0 0) 0x7f4d18003520 con 0x7f4d300ce240 2024-04-28T12:37:05.166 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 >> v2:172.21.15.131:3300/0 conn(0x7f4d300b3810 msgr2=0x7f4d30075730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:05.166 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 --2- 172.21.15.157:0/1419430131 >> v2:172.21.15.131:3300/0 conn(0x7f4d300b3810 0x7f4d30075730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.166 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 >> v1:172.21.15.131:6789/0 conn(0x7f4d3015f1b0 legacy=0x7f4d300cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d300ae350 con 0x7f4d300ce240 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f4d1c001670 con 0x7f4d300ce240 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 >> v1:172.21.15.157:6789/0 conn(0x7f4d300ce240 legacy=0x7f4d300aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 --2- 172.21.15.157:0/1419430131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d18003a20 0x7f4d18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 --2- 172.21.15.157:0/1419430131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d18004800 0x7f4d18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 --2- 172.21.15.157:0/1419430131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d180051c0 0x7f4d1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3015f590 con 0x7f4d18003a20 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1800c090 con 0x7f4d18004800 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d1800c290 con 0x7f4d180051c0 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d2fe00640 1 --2- 172.21.15.157:0/1419430131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d18004800 0x7f4d18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.168+0000 7f4d35600640 1 --2- 172.21.15.157:0/1419430131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d180051c0 0x7f4d1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.169+0000 7f4d34c00640 1 --2- 172.21.15.157:0/1419430131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d18003a20 0x7f4d18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.169+0000 7f4d35600640 1 -- 172.21.15.157:0/1419430131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d18004800 msgr2=0x7f4d18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.169+0000 7f4d35600640 1 --2- 172.21.15.157:0/1419430131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d18004800 0x7f4d18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.167 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.169+0000 7f4d35600640 1 -- 172.21.15.157:0/1419430131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d18003a20 msgr2=0x7f4d18003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:05.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.169+0000 7f4d35600640 1 --2- 172.21.15.157:0/1419430131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d18003a20 0x7f4d18003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.168 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.169+0000 7f4d35600640 1 -- 172.21.15.157:0/1419430131 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d18003520 con 0x7f4d180051c0 2024-04-28T12:37:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.170+0000 7f4d35600640 1 -- 172.21.15.157:0/1419430131 reap_dead start 2024-04-28T12:37:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.170+0000 7f4d35600640 1 --2- 172.21.15.157:0/1419430131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d180051c0 0x7f4d1800b9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009f90 tx=0x7f4d18005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c6a7414302c6a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.170+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28009c10 con 0x7f4d180051c0 2024-04-28T12:37:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.170+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d28031440 con 0x7f4d180051c0 2024-04-28T12:37:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.170+0000 7f4d2f400640 1 -- 172.21.15.157:0/1419430131 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28031810 con 0x7f4d180051c0 2024-04-28T12:37:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.171+0000 7f4d36000640 1 -- 172.21.15.157:0/1419430131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d180051c0 msgr2=0x7f4d1800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.171+0000 7f4d36000640 1 --2- 172.21.15.157:0/1419430131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d180051c0 0x7f4d1800b9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009f90 tx=0x7f4d18005830 comp rx=0 tx=0).stop 2024-04-28T12:37:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.171+0000 7f4d36000640 1 -- 172.21.15.157:0/1419430131 shutdown_connections 2024-04-28T12:37:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.171+0000 7f4d36000640 1 --2- 172.21.15.157:0/1419430131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d180051c0 0x7f4d1800b9c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.171+0000 7f4d36000640 1 -- 172.21.15.157:0/1419430131 >> 172.21.15.157:0/1419430131 conn(0x7f4d300b2cf0 msgr2=0x7f4d300b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.171+0000 7f4d36000640 1 -- 172.21.15.157:0/1419430131 shutdown_connections 2024-04-28T12:37:05.169 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.171+0000 7f4d36000640 1 -- 172.21.15.157:0/1419430131 wait complete. 2024-04-28T12:37:05.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.173+0000 7f4d36000640 1 Processor -- start 2024-04-28T12:37:05.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.173+0000 7f4d36000640 1 -- start start 2024-04-28T12:37:05.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d36000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d180051c0 0x7f4d30153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d36000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d300c5820 0x7f4d301f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d36000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d30159770 0x7f4d30159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d36000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d300cb770 con 0x7f4d180051c0 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d36000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d300ae9a0 con 0x7f4d30159770 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d36000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d300d0e70 con 0x7f4d300c5820 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d2fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d300c5820 0x7f4d301f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d2fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d300c5820 0x7f4d301f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:43922/0 (socket says 172.21.15.157:43922) 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d2fe00640 1 -- 172.21.15.157:0/4056851541 learned_addr learned my addr 172.21.15.157:0/4056851541 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d35600640 1 --2- 172.21.15.157:0/4056851541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d30159770 0x7f4d30159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d2fe00640 1 -- 172.21.15.157:0/4056851541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d300c5820 msgr2=0x7f4d301f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d2fe00640 1 -- 172.21.15.157:0/4056851541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d300c5820 msgr2=0x7f4d301f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d2fe00640 1 --2- 172.21.15.157:0/4056851541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d300c5820 0x7f4d301f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d34c00640 1 --2- 172.21.15.157:0/4056851541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d180051c0 0x7f4d30153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d2fe00640 1 --2- 172.21.15.157:0/4056851541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d300c5820 0x7f4d301f4520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:05.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d35600640 1 -- 172.21.15.157:0/4056851541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d300c5820 msgr2=0x7f4d301f4520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:37:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d35600640 1 --2- 172.21.15.157:0/4056851541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d300c5820 0x7f4d301f4520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d35600640 1 -- 172.21.15.157:0/4056851541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d180051c0 msgr2=0x7f4d30153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d35600640 1 --2- 172.21.15.157:0/4056851541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d180051c0 0x7f4d30153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.174+0000 7f4d35600640 1 -- 172.21.15.157:0/4056851541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d300b6ed0 con 0x7f4d30159770 2024-04-28T12:37:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.175+0000 7f4d35600640 1 --2- 172.21.15.157:0/4056851541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d30159770 0x7f4d30159bf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4d300b2cc0 tx=0x7f4d28009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.175+0000 7f4d2cc00640 1 -- 172.21.15.157:0/4056851541 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28009c10 con 0x7f4d30159770 2024-04-28T12:37:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.175+0000 7f4d2cc00640 1 -- 172.21.15.157:0/4056851541 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d28043070 con 0x7f4d30159770 2024-04-28T12:37:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.175+0000 7f4d2cc00640 1 -- 172.21.15.157:0/4056851541 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2803c860 con 0x7f4d30159770 2024-04-28T12:37:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.175+0000 7f4d36000640 1 -- 172.21.15.157:0/4056851541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d300c4d40 con 0x7f4d30159770 2024-04-28T12:37:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.175+0000 7f4d36000640 1 -- 172.21.15.157:0/4056851541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d30159490 con 0x7f4d30159770 2024-04-28T12:37:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.175+0000 7f4d2cc00640 1 -- 172.21.15.157:0/4056851541 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4d28005d10 con 0x7f4d30159770 2024-04-28T12:37:05.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.175+0000 7f4d2cc00640 1 -- 172.21.15.157:0/4056851541 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f4d280473c0 con 0x7f4d30159770 2024-04-28T12:37:05.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.176+0000 7f4d36000640 1 -- 172.21.15.157:0/4056851541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d3006d040 con 0x7f4d30159770 2024-04-28T12:37:05.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.178+0000 7f4d2cc00640 1 -- 172.21.15.157:0/4056851541 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4d3006d040 con 0x7f4d30159770 2024-04-28T12:37:05.187 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:05.190+0000 7f51361ee040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-28T12:37:05.233 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:05.237+0000 7fb17f1bc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:37:05.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.254+0000 7f4d36000640 1 -- 172.21.15.157:0/4056851541 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6", "id": 4} v 0) -- 0x7f4d300506c0 con 0x7f4d30159770 2024-04-28T12:37:05.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.262+0000 7f4d2cc00640 1 -- 172.21.15.157:0/4056851541 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f4d2803ca00 con 0x7f4d30159770 2024-04-28T12:37:05.259 INFO:teuthology.orchestra.run.smithi157.stdout:4 2024-04-28T12:37:05.260 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.263+0000 7f4d36000640 1 -- 172.21.15.157:0/4056851541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d30159770 msgr2=0x7f4d30159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:05.260 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.263+0000 7f4d36000640 1 --2- 172.21.15.157:0/4056851541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d30159770 0x7f4d30159bf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4d300b2cc0 tx=0x7f4d28009750 comp rx=0 tx=0).stop 2024-04-28T12:37:05.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.264+0000 7f4d36000640 1 -- 172.21.15.157:0/4056851541 shutdown_connections 2024-04-28T12:37:05.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.264+0000 7f4d36000640 1 --2- 172.21.15.157:0/4056851541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d30159770 0x7f4d30159bf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.264+0000 7f4d36000640 1 --2- 172.21.15.157:0/4056851541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d300c5820 0x7f4d301f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.264+0000 7f4d36000640 1 --2- 172.21.15.157:0/4056851541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d180051c0 0x7f4d30153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.264+0000 7f4d36000640 1 -- 172.21.15.157:0/4056851541 >> 172.21.15.157:0/4056851541 conn(0x7f4d300b2cf0 msgr2=0x7f4d300ca200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:05.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.264+0000 7f4d36000640 1 -- 172.21.15.157:0/4056851541 shutdown_connections 2024-04-28T12:37:05.261 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.264+0000 7f4d36000640 1 -- 172.21.15.157:0/4056851541 wait complete. 2024-04-28T12:37:05.277 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 7549d7c2-0964-4a6f-abe9-a0ba585e1267 5 2024-04-28T12:37:05.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.449+0000 7f1f98e00640 1 Processor -- start 2024-04-28T12:37:05.446 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.449+0000 7f1f98e00640 1 -- start start 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.451+0000 7f1f98e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f940afe90 0x7f1f94073370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.451+0000 7f1f98e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f9406c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.451+0000 7f1f98e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f940d1b50 0x7f1f9406c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.451+0000 7f1f98e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1f940b0760 con 0x7f1f940afe90 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.451+0000 7f1f98e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1f940b09b0 con 0x7f1f940d1b50 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.451+0000 7f1f98e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1f940b0c90 con 0x7f1f940d16d0 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.451+0000 7f1f93e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f940d1b50 0x7f1f9406c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.451+0000 7f1f92a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f9406c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.451+0000 7f1f93e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f940d1b50 0x7f1f9406c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39524/0 (socket says 172.21.15.157:39524) 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.451+0000 7f1f93e00640 1 -- 172.21.15.157:0/2318102470 learned_addr learned my addr 172.21.15.157:0/2318102470 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.452+0000 7f1f92a00640 1 -- 172.21.15.157:0/2318102470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 msgr2=0x7f1f9406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.452+0000 7f1f92a00640 1 -- 172.21.15.157:0/2318102470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 msgr2=0x7f1f9406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:05.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.452+0000 7f1f93400640 1 --2- 172.21.15.157:0/2318102470 >> v2:172.21.15.131:3300/0 conn(0x7f1f940afe90 0x7f1f94073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.452+0000 7f1f92a00640 1 --2- 172.21.15.157:0/2318102470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f9406c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:05.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.452+0000 7f1f92a00640 1 --2- 172.21.15.157:0/2318102470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f9406c350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:05.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.452+0000 7f1f93e00640 1 -- 172.21.15.157:0/2318102470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 msgr2=0x7f1f9406c350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:37:05.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.452+0000 7f1f93e00640 1 --2- 172.21.15.157:0/2318102470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f9406c350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.452+0000 7f1f93e00640 1 -- 172.21.15.157:0/2318102470 >> v2:172.21.15.131:3300/0 conn(0x7f1f940afe90 msgr2=0x7f1f94073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:05.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.452+0000 7f1f93e00640 1 --2- 172.21.15.157:0/2318102470 >> v2:172.21.15.131:3300/0 conn(0x7f1f940afe90 0x7f1f94073370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.452+0000 7f1f93e00640 1 -- 172.21.15.157:0/2318102470 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f940b0f70 con 0x7f1f940d1b50 2024-04-28T12:37:05.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.453+0000 7f1f93e00640 1 --2- 172.21.15.157:0/2318102470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f940d1b50 0x7f1f9406c890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1f94068eb0 tx=0x7f1f8c02a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b29901a7efbfc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:05.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.453+0000 7f1f92000640 1 -- 172.21.15.157:0/2318102470 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c036070 con 0x7f1f940d1b50 2024-04-28T12:37:05.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.453+0000 7f1f92000640 1 -- 172.21.15.157:0/2318102470 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c031600 con 0x7f1f940d1b50 2024-04-28T12:37:05.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.453+0000 7f1f92000640 1 -- 172.21.15.157:0/2318102470 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0319d0 con 0x7f1f940d1b50 2024-04-28T12:37:05.451 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.453+0000 7f1f98e00640 1 -- 172.21.15.157:0/2318102470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f940d1b50 msgr2=0x7f1f9406c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:05.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.453+0000 7f1f98e00640 1 --2- 172.21.15.157:0/2318102470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f940d1b50 0x7f1f9406c890 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1f94068eb0 tx=0x7f1f8c02a6a0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.454+0000 7f1f98e00640 1 -- 172.21.15.157:0/2318102470 shutdown_connections 2024-04-28T12:37:05.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.454+0000 7f1f98e00640 1 --2- 172.21.15.157:0/2318102470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f940d1b50 0x7f1f9406c890 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.454+0000 7f1f98e00640 1 --2- 172.21.15.157:0/2318102470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f9406c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.454+0000 7f1f98e00640 1 --2- 172.21.15.157:0/2318102470 >> v2:172.21.15.131:3300/0 conn(0x7f1f940afe90 0x7f1f94073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.454+0000 7f1f98e00640 1 -- 172.21.15.157:0/2318102470 >> 172.21.15.157:0/2318102470 conn(0x7f1f94068090 msgr2=0x7f1f94068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:05.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.454+0000 7f1f98e00640 1 -- 172.21.15.157:0/2318102470 shutdown_connections 2024-04-28T12:37:05.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.454+0000 7f1f98e00640 1 -- 172.21.15.157:0/2318102470 wait complete. 2024-04-28T12:37:05.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.455+0000 7f1f98e00640 1 Processor -- start 2024-04-28T12:37:05.453 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.456+0000 7f1f98e00640 1 -- start start 2024-04-28T12:37:05.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.456+0000 7f1f98e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f940afe90 0x7f1f94157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.456+0000 7f1f98e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f941f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.455 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f98e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f940d1b50 0x7f1f941e8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f98e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1f940b0270 con 0x7f1f940afe90 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f98e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94070910 con 0x7f1f940d1b50 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f98e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1f940738b0 con 0x7f1f940d16d0 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f93e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f940d1b50 0x7f1f941e8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f92a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f941f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f92a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f941f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:43946/0 (socket says 172.21.15.157:43946) 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f92a00640 1 -- 172.21.15.157:0/1509747705 learned_addr learned my addr 172.21.15.157:0/1509747705 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f93400640 1 --2- 172.21.15.157:0/1509747705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f940afe90 0x7f1f94157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f92a00640 1 -- 172.21.15.157:0/1509747705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 msgr2=0x7f1f941f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f92a00640 1 -- 172.21.15.157:0/1509747705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 msgr2=0x7f1f941f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f92a00640 1 --2- 172.21.15.157:0/1509747705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f941f5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f92a00640 1 --2- 172.21.15.157:0/1509747705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f941f5cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f93e00640 1 -- 172.21.15.157:0/1509747705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 msgr2=0x7f1f941f5cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:37:05.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f93e00640 1 --2- 172.21.15.157:0/1509747705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f941f5cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f93e00640 1 -- 172.21.15.157:0/1509747705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f940afe90 msgr2=0x7f1f94157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f93e00640 1 --2- 172.21.15.157:0/1509747705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f940afe90 0x7f1f94157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f93e00640 1 -- 172.21.15.157:0/1509747705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f941e80a0 con 0x7f1f940d1b50 2024-04-28T12:37:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.457+0000 7f1f93e00640 1 --2- 172.21.15.157:0/1509747705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f940d1b50 0x7f1f941e8e60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c030000 tx=0x7f1f8c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.458+0000 7f1f8b400640 1 -- 172.21.15.157:0/1509747705 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c036050 con 0x7f1f940d1b50 2024-04-28T12:37:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.458+0000 7f1f8b400640 1 -- 172.21.15.157:0/1509747705 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c042070 con 0x7f1f940d1b50 2024-04-28T12:37:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.458+0000 7f1f98e00640 1 -- 172.21.15.157:0/1509747705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f941e8380 con 0x7f1f940d1b50 2024-04-28T12:37:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.458+0000 7f1f8b400640 1 -- 172.21.15.157:0/1509747705 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c009510 con 0x7f1f940d1b50 2024-04-28T12:37:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.458+0000 7f1f98e00640 1 -- 172.21.15.157:0/1509747705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f941e8940 con 0x7f1f940d1b50 2024-04-28T12:37:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.458+0000 7f1f8b400640 1 -- 172.21.15.157:0/1509747705 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1f8c007bd0 con 0x7f1f940d1b50 2024-04-28T12:37:05.457 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.458+0000 7f1f8b400640 1 -- 172.21.15.157:0/1509747705 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f1f8c0479e0 con 0x7f1f940d1b50 2024-04-28T12:37:05.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.459+0000 7f1f98e00640 1 -- 172.21.15.157:0/1509747705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f940506c0 con 0x7f1f940d1b50 2024-04-28T12:37:05.458 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.461+0000 7f1f8b400640 1 -- 172.21.15.157:0/1509747705 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1f8c047460 con 0x7f1f940d1b50 2024-04-28T12:37:05.530 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:05.533+0000 7f51361ee040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-28T12:37:05.538 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.542+0000 7f1f98e00640 1 -- 172.21.15.157:0/1509747705 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7549d7c2-0964-4a6f-abe9-a0ba585e1267", "id": 5} v 0) -- 0x7f1f941e8b10 con 0x7f1f940d1b50 2024-04-28T12:37:05.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.549+0000 7f1f8b400640 1 -- 172.21.15.157:0/1509747705 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7549d7c2-0964-4a6f-abe9-a0ba585e1267", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f1f8c047640 con 0x7f1f940d1b50 2024-04-28T12:37:05.546 INFO:teuthology.orchestra.run.smithi157.stdout:5 2024-04-28T12:37:05.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.550+0000 7f1f98e00640 1 -- 172.21.15.157:0/1509747705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f940d1b50 msgr2=0x7f1f941e8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:05.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.550+0000 7f1f98e00640 1 --2- 172.21.15.157:0/1509747705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f940d1b50 0x7f1f941e8e60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c030000 tx=0x7f1f8c004a40 comp rx=0 tx=0).stop 2024-04-28T12:37:05.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.550+0000 7f1f98e00640 1 -- 172.21.15.157:0/1509747705 shutdown_connections 2024-04-28T12:37:05.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.550+0000 7f1f98e00640 1 --2- 172.21.15.157:0/1509747705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f940d1b50 0x7f1f941e8e60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.550+0000 7f1f98e00640 1 --2- 172.21.15.157:0/1509747705 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f940d16d0 0x7f1f941f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.550+0000 7f1f98e00640 1 --2- 172.21.15.157:0/1509747705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f940afe90 0x7f1f94157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.550+0000 7f1f98e00640 1 -- 172.21.15.157:0/1509747705 >> 172.21.15.157:0/1509747705 conn(0x7f1f94068090 msgr2=0x7f1f941556b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:05.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.551+0000 7f1f98e00640 1 -- 172.21.15.157:0/1509747705 shutdown_connections 2024-04-28T12:37:05.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.551+0000 7f1f98e00640 1 -- 172.21.15.157:0/1509747705 wait complete. 2024-04-28T12:37:05.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 7570ea49-d73f-4998-a6b5-e0cfa5162fe6 6 2024-04-28T12:37:05.590 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:05.593+0000 7f51361ee040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-28T12:37:05.684 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:05.687+0000 7fb17f1bc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-28T12:37:05.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.743+0000 7faad2400640 1 Processor -- start 2024-04-28T12:37:05.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.743+0000 7faad2400640 1 -- start start 2024-04-28T12:37:05.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.745+0000 7faad2400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faacc15ef30 0x7faacc0d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.745+0000 7faad2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc15f310 0x7faacc0d3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.745+0000 7faad2400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faacc0d3a70 con 0x7faacc0d0600 2024-04-28T12:37:05.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.745+0000 7faad2400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faacc0d3840 con 0x7faacc15ef30 2024-04-28T12:37:05.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.745+0000 7faad2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faacc0cfb20 con 0x7faacc15f310 2024-04-28T12:37:05.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.745+0000 7faad1000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faacc15ef30 0x7faacc0d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.745+0000 7faad1a00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7faacc0d0600 0x7faacc162c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.157:53512/0 (socket says 172.21.15.157:53512) 2024-04-28T12:37:05.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.745+0000 7faad1a00640 1 -- 172.21.15.157:0/1297525776 learned_addr learned my addr 172.21.15.157:0/1297525776 (peer_addr_for_me v1:172.21.15.157:0/0) 2024-04-28T12:37:05.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.745+0000 7faacbe00640 1 --2- 172.21.15.157:0/1297525776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc15f310 0x7faacc0d3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.746+0000 7faacbe00640 1 -- 172.21.15.157:0/1297525776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc15f310 msgr2=0x7faacc0d3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T12:37:05.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.746+0000 7faacbe00640 1 -- 172.21.15.157:0/1297525776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc15f310 msgr2=0x7faacc0d3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:05.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.746+0000 7faacbe00640 1 --2- 172.21.15.157:0/1297525776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc15f310 0x7faacc0d3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:05.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.746+0000 7faacbe00640 1 --2- 172.21.15.157:0/1297525776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc15f310 0x7faacc0d3300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:05.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.746+0000 7faad1000640 1 -- 172.21.15.157:0/1297525776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc15f310 msgr2=0x7faacc0d3300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:37:05.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.746+0000 7faad1000640 1 --2- 172.21.15.157:0/1297525776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc15f310 0x7faacc0d3300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.746+0000 7faad1000640 1 -- 172.21.15.157:0/1297525776 >> v1:172.21.15.157:6789/0 conn(0x7faacc0d0600 legacy=0x7faacc162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:05.744 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.746+0000 7faad1000640 1 -- 172.21.15.157:0/1297525776 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faacc0cfe00 con 0x7faacc15ef30 2024-04-28T12:37:05.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.747+0000 7faad1000640 1 --2- 172.21.15.157:0/1297525776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faacc15ef30 0x7faacc0d5810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faac00097a0 tx=0x7faac002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e196a94a4e3186ec server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:05.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.747+0000 7faacb400640 1 -- 172.21.15.157:0/1297525776 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0037070 con 0x7faacc15ef30 2024-04-28T12:37:05.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.747+0000 7faacb400640 1 -- 172.21.15.157:0/1297525776 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac002ae50 con 0x7faacc15ef30 2024-04-28T12:37:05.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.747+0000 7faacb400640 1 -- 172.21.15.157:0/1297525776 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0032660 con 0x7faacc15ef30 2024-04-28T12:37:05.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.748+0000 7faad2400640 1 -- 172.21.15.157:0/1297525776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faacc15ef30 msgr2=0x7faacc0d5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:05.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.748+0000 7faad2400640 1 --2- 172.21.15.157:0/1297525776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faacc15ef30 0x7faacc0d5810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faac00097a0 tx=0x7faac002a100 comp rx=0 tx=0).stop 2024-04-28T12:37:05.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.748+0000 7faad2400640 1 -- 172.21.15.157:0/1297525776 shutdown_connections 2024-04-28T12:37:05.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.748+0000 7faad2400640 1 --2- 172.21.15.157:0/1297525776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc15f310 0x7faacc0d3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.748+0000 7faad2400640 1 --2- 172.21.15.157:0/1297525776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faacc15ef30 0x7faacc0d5810 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.748+0000 7faad2400640 1 -- 172.21.15.157:0/1297525776 >> 172.21.15.157:0/1297525776 conn(0x7faacc0d4d20 msgr2=0x7faacc0d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:05.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.748+0000 7faad2400640 1 -- 172.21.15.157:0/1297525776 shutdown_connections 2024-04-28T12:37:05.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.748+0000 7faad2400640 1 -- 172.21.15.157:0/1297525776 wait complete. 2024-04-28T12:37:05.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.749+0000 7faad2400640 1 Processor -- start 2024-04-28T12:37:05.746 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.750+0000 7faad2400640 1 -- start start 2024-04-28T12:37:05.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.750+0000 7faad2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc0d0600 0x7faacc0c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.750+0000 7faad2400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faacc15ef30 0x7faacc1614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.750+0000 7faad2400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc161da0 0x7faacc0cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:05.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faad2400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faacc163310 con 0x7faacc161da0 2024-04-28T12:37:05.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faad2400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faacc0cf7f0 con 0x7faacc15ef30 2024-04-28T12:37:05.749 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faad2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faacc163490 con 0x7faacc0d0600 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faad1000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc0d0600 0x7faacc0c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faacbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faacc15ef30 0x7faacc1614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faad1000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc0d0600 0x7faacc0c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:43964/0 (socket says 172.21.15.157:43964) 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faad1000640 1 -- 172.21.15.157:0/3215230940 learned_addr learned my addr 172.21.15.157:0/3215230940 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faad1000640 1 -- 172.21.15.157:0/3215230940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc0d0600 msgr2=0x7faacc0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faad1000640 1 -- 172.21.15.157:0/3215230940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc0d0600 msgr2=0x7faacc0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faad1000640 1 --2- 172.21.15.157:0/3215230940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc0d0600 0x7faacc0c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faad1000640 1 --2- 172.21.15.157:0/3215230940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc0d0600 0x7faacc0c9260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faacbe00640 1 -- 172.21.15.157:0/3215230940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc0d0600 msgr2=0x7faacc0c9260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faacbe00640 1 --2- 172.21.15.157:0/3215230940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc0d0600 0x7faacc0c9260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faacbe00640 1 -- 172.21.15.157:0/3215230940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc161da0 msgr2=0x7faacc0cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faad1a00640 1 --2- 172.21.15.157:0/3215230940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc161da0 0x7faacc0cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:05.750 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faacbe00640 1 --2- 172.21.15.157:0/3215230940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc161da0 0x7faacc0cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faacbe00640 1 -- 172.21.15.157:0/3215230940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faacc162510 con 0x7faacc15ef30 2024-04-28T12:37:05.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.751+0000 7faacbe00640 1 --2- 172.21.15.157:0/3215230940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faacc15ef30 0x7faacc1614f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faab800d5d0 tx=0x7faab800da90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:05.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.752+0000 7faac8c00640 1 -- 172.21.15.157:0/3215230940 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab80125f0 con 0x7faacc15ef30 2024-04-28T12:37:05.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.752+0000 7faac8c00640 1 -- 172.21.15.157:0/3215230940 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab8019070 con 0x7faacc15ef30 2024-04-28T12:37:05.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.752+0000 7faad2400640 1 -- 172.21.15.157:0/3215230940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faacc162230 con 0x7faacc15ef30 2024-04-28T12:37:05.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.752+0000 7faac8c00640 1 -- 172.21.15.157:0/3215230940 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab80129d0 con 0x7faacc15ef30 2024-04-28T12:37:05.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.752+0000 7faad2400640 1 -- 172.21.15.157:0/3215230940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faacc160cf0 con 0x7faacc15ef30 2024-04-28T12:37:05.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.752+0000 7faac8c00640 1 -- 172.21.15.157:0/3215230940 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faab8012b70 con 0x7faacc15ef30 2024-04-28T12:37:05.751 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.752+0000 7faac8c00640 1 -- 172.21.15.157:0/3215230940 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7faab801ea50 con 0x7faacc15ef30 2024-04-28T12:37:05.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.753+0000 7faad2400640 1 -- 172.21.15.157:0/3215230940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faacc0506c0 con 0x7faacc15ef30 2024-04-28T12:37:05.752 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.755+0000 7faac8c00640 1 -- 172.21.15.157:0/3215230940 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7faab801e460 con 0x7faacc15ef30 2024-04-28T12:37:05.781 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:05.784+0000 7f51361ee040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-28T12:37:05.831 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.835+0000 7faad2400640 1 -- 172.21.15.157:0/3215230940 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7570ea49-d73f-4998-a6b5-e0cfa5162fe6", "id": 6} v 0) -- 0x7faacc06dfe0 con 0x7faacc15ef30 2024-04-28T12:37:05.839 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.842+0000 7faac8c00640 1 -- 172.21.15.157:0/3215230940 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7570ea49-d73f-4998-a6b5-e0cfa5162fe6", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7faab801e710 con 0x7faacc15ef30 2024-04-28T12:37:05.839 INFO:teuthology.orchestra.run.smithi157.stdout:6 2024-04-28T12:37:05.840 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.843+0000 7faad2400640 1 -- 172.21.15.157:0/3215230940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faacc15ef30 msgr2=0x7faacc1614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:05.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.843+0000 7faad2400640 1 --2- 172.21.15.157:0/3215230940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faacc15ef30 0x7faacc1614f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faab800d5d0 tx=0x7faab800da90 comp rx=0 tx=0).stop 2024-04-28T12:37:05.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.844+0000 7faad2400640 1 -- 172.21.15.157:0/3215230940 shutdown_connections 2024-04-28T12:37:05.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.844+0000 7faad2400640 1 --2- 172.21.15.157:0/3215230940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faacc161da0 0x7faacc0cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.844+0000 7faad2400640 1 --2- 172.21.15.157:0/3215230940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faacc15ef30 0x7faacc1614f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.844+0000 7faad2400640 1 --2- 172.21.15.157:0/3215230940 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faacc0d0600 0x7faacc0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:05.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.844+0000 7faad2400640 1 -- 172.21.15.157:0/3215230940 >> 172.21.15.157:0/3215230940 conn(0x7faacc0d4d20 msgr2=0x7faacc074f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:05.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.844+0000 7faad2400640 1 -- 172.21.15.157:0/3215230940 shutdown_connections 2024-04-28T12:37:05.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:05.844+0000 7faad2400640 1 -- 172.21.15.157:0/3215230940 wait complete. 2024-04-28T12:37:05.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 6c98a122-d9fc-43fa-8b4e-9934e54e369d 7 2024-04-28T12:37:05.956 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:05.960+0000 7f51361ee040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-28T12:37:05.978 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:05.981+0000 7fb17f1bc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-28T12:37:06.018 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:06.021+0000 7f51361ee040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-28T12:37:06.039 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:06.042+0000 7fb17f1bc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-28T12:37:06.079 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.082+0000 7fb41b200640 1 Processor -- start 2024-04-28T12:37:06.079 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.082+0000 7fb41b200640 1 -- start start 2024-04-28T12:37:06.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb41b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:06.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb41b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb41415f310 0x7fb4140d3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:06.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb41b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb4140d0600 0x7fb414162c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:06.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb41b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb4140d3840 con 0x7fb4140d0600 2024-04-28T12:37:06.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb41b200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb4140d3a70 con 0x7fb41415f310 2024-04-28T12:37:06.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb41b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb4140cfb20 con 0x7fb41415ef30 2024-04-28T12:37:06.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb419e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:06.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb419400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb41415f310 0x7fb4140d3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:06.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb419e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140d5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:43966/0 (socket says 172.21.15.157:43966) 2024-04-28T12:37:06.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb419e00640 1 -- 172.21.15.157:0/361697426 learned_addr learned my addr 172.21.15.157:0/361697426 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:06.082 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb419e00640 1 -- 172.21.15.157:0/361697426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 msgr2=0x7fb4140d5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-28T12:37:06.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb419e00640 1 -- 172.21.15.157:0/361697426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 msgr2=0x7fb4140d5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:06.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.084+0000 7fb419e00640 1 --2- 172.21.15.157:0/361697426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140d5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:06.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.085+0000 7fb419e00640 1 --2- 172.21.15.157:0/361697426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140d5810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:06.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.085+0000 7fb41a800640 1 --2- 172.21.15.157:0/361697426 >> v2:172.21.15.131:3300/0 conn(0x7fb4140d0600 0x7fb414162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:06.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.085+0000 7fb419400640 1 -- 172.21.15.157:0/361697426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 msgr2=0x7fb4140d5810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:37:06.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.085+0000 7fb419400640 1 --2- 172.21.15.157:0/361697426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140d5810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.085+0000 7fb419400640 1 -- 172.21.15.157:0/361697426 >> v2:172.21.15.131:3300/0 conn(0x7fb4140d0600 msgr2=0x7fb414162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:06.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.085+0000 7fb419400640 1 --2- 172.21.15.157:0/361697426 >> v2:172.21.15.131:3300/0 conn(0x7fb4140d0600 0x7fb414162c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.083 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.085+0000 7fb419400640 1 -- 172.21.15.157:0/361697426 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4140cfe00 con 0x7fb41415f310 2024-04-28T12:37:06.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.086+0000 7fb419400640 1 --2- 172.21.15.157:0/361697426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb41415f310 0x7fb4140d3300 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb41408d360 tx=0x7fb40402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24b94f6cf136657a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:06.084 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.086+0000 7fb418a00640 1 -- 172.21.15.157:0/361697426 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404037070 con 0x7fb41415f310 2024-04-28T12:37:06.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.086+0000 7fb418a00640 1 -- 172.21.15.157:0/361697426 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40402add0 con 0x7fb41415f310 2024-04-28T12:37:06.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.086+0000 7fb418a00640 1 -- 172.21.15.157:0/361697426 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404032660 con 0x7fb41415f310 2024-04-28T12:37:06.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.087+0000 7fb41b200640 1 -- 172.21.15.157:0/361697426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb41415f310 msgr2=0x7fb4140d3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:06.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.087+0000 7fb41b200640 1 --2- 172.21.15.157:0/361697426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb41415f310 0x7fb4140d3300 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb41408d360 tx=0x7fb40402a080 comp rx=0 tx=0).stop 2024-04-28T12:37:06.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.087+0000 7fb41b200640 1 -- 172.21.15.157:0/361697426 shutdown_connections 2024-04-28T12:37:06.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.087+0000 7fb41b200640 1 --2- 172.21.15.157:0/361697426 >> v2:172.21.15.131:3300/0 conn(0x7fb4140d0600 0x7fb414162c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.087+0000 7fb41b200640 1 --2- 172.21.15.157:0/361697426 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb41415f310 0x7fb4140d3300 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.087+0000 7fb41b200640 1 --2- 172.21.15.157:0/361697426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140d5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.087+0000 7fb41b200640 1 -- 172.21.15.157:0/361697426 >> 172.21.15.157:0/361697426 conn(0x7fb4140d4d20 msgr2=0x7fb4140d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:06.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.087+0000 7fb41b200640 1 -- 172.21.15.157:0/361697426 shutdown_connections 2024-04-28T12:37:06.085 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.087+0000 7fb41b200640 1 -- 172.21.15.157:0/361697426 wait complete. 2024-04-28T12:37:06.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.089+0000 7fb41b200640 1 Processor -- start 2024-04-28T12:37:06.086 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.089+0000 7fb41b200640 1 -- start start 2024-04-28T12:37:06.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.090+0000 7fb41b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4140d0600 0x7fb414160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:06.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.090+0000 7fb41b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:06.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.090+0000 7fb41b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4141f58b0 0x7fb41415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:06.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.090+0000 7fb41b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb4140d0a70 con 0x7fb4140d0600 2024-04-28T12:37:06.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.090+0000 7fb41b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb4140cf7f0 con 0x7fb4141f58b0 2024-04-28T12:37:06.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.090+0000 7fb41b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb4140d0bf0 con 0x7fb41415ef30 2024-04-28T12:37:06.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb419400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:06.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb41a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4141f58b0 0x7fb41415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:06.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb41a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4141f58b0 0x7fb41415be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:39548/0 (socket says 172.21.15.157:39548) 2024-04-28T12:37:06.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb419400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:43968/0 (socket says 172.21.15.157:43968) 2024-04-28T12:37:06.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb41a800640 1 -- 172.21.15.157:0/1189257053 learned_addr learned my addr 172.21.15.157:0/1189257053 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-28T12:37:06.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb419e00640 1 --2- 172.21.15.157:0/1189257053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4140d0600 0x7fb414160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:06.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb419400640 1 -- 172.21.15.157:0/1189257053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 msgr2=0x7fb4140c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-28T12:37:06.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb419400640 1 -- 172.21.15.157:0/1189257053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 msgr2=0x7fb4140c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:37:06.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb419400640 1 --2- 172.21.15.157:0/1189257053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:06.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb419400640 1 --2- 172.21.15.157:0/1189257053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140c9470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:06.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb41a800640 1 -- 172.21.15.157:0/1189257053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 msgr2=0x7fb4140c9470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:37:06.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb41a800640 1 --2- 172.21.15.157:0/1189257053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140c9470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb41a800640 1 -- 172.21.15.157:0/1189257053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4140d0600 msgr2=0x7fb414160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:06.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb41a800640 1 --2- 172.21.15.157:0/1189257053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4140d0600 0x7fb414160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.091+0000 7fb41a800640 1 -- 172.21.15.157:0/1189257053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4140c9ce0 con 0x7fb4141f58b0 2024-04-28T12:37:06.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.092+0000 7fb41a800640 1 --2- 172.21.15.157:0/1189257053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4141f58b0 0x7fb41415be00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb410007e60 tx=0x7fb41000e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:06.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.092+0000 7fb40e000640 1 -- 172.21.15.157:0/1189257053 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410014420 con 0x7fb4141f58b0 2024-04-28T12:37:06.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.092+0000 7fb40e000640 1 -- 172.21.15.157:0/1189257053 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb410019070 con 0x7fb4141f58b0 2024-04-28T12:37:06.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.092+0000 7fb40e000640 1 -- 172.21.15.157:0/1189257053 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410014820 con 0x7fb4141f58b0 2024-04-28T12:37:06.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.092+0000 7fb41b200640 1 -- 172.21.15.157:0/1189257053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41415f8a0 con 0x7fb4141f58b0 2024-04-28T12:37:06.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.092+0000 7fb41b200640 1 -- 172.21.15.157:0/1189257053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41415fe60 con 0x7fb4141f58b0 2024-04-28T12:37:06.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.092+0000 7fb40e000640 1 -- 172.21.15.157:0/1189257053 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb4100149c0 con 0x7fb4141f58b0 2024-04-28T12:37:06.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.092+0000 7fb40e000640 1 -- 172.21.15.157:0/1189257053 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fb410014cb0 con 0x7fb4141f58b0 2024-04-28T12:37:06.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.093+0000 7fb41b200640 1 -- 172.21.15.157:0/1189257053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4140506c0 con 0x7fb4141f58b0 2024-04-28T12:37:06.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.095+0000 7fb40e000640 1 -- 172.21.15.157:0/1189257053 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb41001e460 con 0x7fb4141f58b0 2024-04-28T12:37:06.104 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:06.107+0000 7f51361ee040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-28T12:37:06.159 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:06.162+0000 7f51361ee040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-28T12:37:06.169 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:06.172+0000 7fb17f1bc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-28T12:37:06.171 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.175+0000 7fb41b200640 1 -- 172.21.15.157:0/1189257053 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6c98a122-d9fc-43fa-8b4e-9934e54e369d", "id": 7} v 0) -- 0x7fb414075aa0 con 0x7fb4141f58b0 2024-04-28T12:37:06.182 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.182+0000 7fb40e000640 1 -- 172.21.15.157:0/1189257053 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6c98a122-d9fc-43fa-8b4e-9934e54e369d", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fb41001e640 con 0x7fb4141f58b0 2024-04-28T12:37:06.182 INFO:teuthology.orchestra.run.smithi157.stdout:7 2024-04-28T12:37:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.184+0000 7fb41b200640 1 -- 172.21.15.157:0/1189257053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4141f58b0 msgr2=0x7fb41415be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.184+0000 7fb41b200640 1 --2- 172.21.15.157:0/1189257053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4141f58b0 0x7fb41415be00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb410007e60 tx=0x7fb41000e4b0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.184+0000 7fb41b200640 1 -- 172.21.15.157:0/1189257053 shutdown_connections 2024-04-28T12:37:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.184+0000 7fb41b200640 1 --2- 172.21.15.157:0/1189257053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb4141f58b0 0x7fb41415be00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.184+0000 7fb41b200640 1 --2- 172.21.15.157:0/1189257053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb41415ef30 0x7fb4140c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.184+0000 7fb41b200640 1 --2- 172.21.15.157:0/1189257053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4140d0600 0x7fb414160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.184+0000 7fb41b200640 1 -- 172.21.15.157:0/1189257053 >> 172.21.15.157:0/1189257053 conn(0x7fb4140d4d20 msgr2=0x7fb4140b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:06.183 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.184+0000 7fb41b200640 1 -- 172.21.15.157:0/1189257053 shutdown_connections 2024-04-28T12:37:06.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:37:06.184+0000 7fb41b200640 1 -- 172.21.15.157:0/1189257053 wait complete. 2024-04-28T12:37:06.196 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-28T12:37:06.197 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-28T12:37:06.204 INFO:tasks.ceph.osd.0:Started 2024-04-28T12:37:06.204 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-28T12:37:06.204 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-28T12:37:06.213 INFO:tasks.ceph.osd.1:Started 2024-04-28T12:37:06.213 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-28T12:37:06.213 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-28T12:37:06.219 INFO:tasks.ceph.osd.2:Started 2024-04-28T12:37:06.219 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-28T12:37:06.219 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-28T12:37:06.224 INFO:tasks.ceph.osd.3:Started 2024-04-28T12:37:06.224 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-28T12:37:06.224 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-28T12:37:06.231 INFO:tasks.ceph.osd.4:Started 2024-04-28T12:37:06.231 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-28T12:37:06.231 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-28T12:37:06.237 INFO:tasks.ceph.osd.5:Started 2024-04-28T12:37:06.238 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-28T12:37:06.238 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-28T12:37:06.243 INFO:tasks.ceph.osd.6:Started 2024-04-28T12:37:06.244 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-28T12:37:06.244 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-28T12:37:06.249 INFO:tasks.ceph.osd.7:Started 2024-04-28T12:37:06.249 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:06.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.470+0000 7f9cffe00640 1 Processor -- start 2024-04-28T12:37:06.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.470+0000 7f9cffe00640 1 -- start start 2024-04-28T12:37:06.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.471+0000 7f9cffe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d000d1620 0x7f9d000d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:06.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.471+0000 7f9cffe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1a00 0x7f9d000b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:06.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.471+0000 7f9cffe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d000d0480 con 0x7f9d000b50b0 2024-04-28T12:37:06.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.471+0000 7f9cffe00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9d000d0ce0 con 0x7f9d000d1620 2024-04-28T12:37:06.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.471+0000 7f9cffe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9d000d0720 con 0x7f9d000d1a00 2024-04-28T12:37:06.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.471+0000 7f9cfea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9d000d1620 0x7f9d000d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:06.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.471+0000 7f9cff400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9d000b50b0 0x7f9d000b5540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33602/0 (socket says 172.21.15.131:33602) 2024-04-28T12:37:06.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.471+0000 7f9cff400640 1 -- 172.21.15.131:0/2045484089 learned_addr learned my addr 172.21.15.131:0/2045484089 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:06.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.472+0000 7f9cfe000640 1 --2- 172.21.15.131:0/2045484089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1a00 0x7f9d000b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:06.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.472+0000 7f9cfe000640 1 -- 172.21.15.131:0/2045484089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1a00 msgr2=0x7f9d000b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-28T12:37:06.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.472+0000 7f9cfe000640 1 -- 172.21.15.131:0/2045484089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1a00 msgr2=0x7f9d000b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-28T12:37:06.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.472+0000 7f9cfe000640 1 --2- 172.21.15.131:0/2045484089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1a00 0x7f9d000b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:06.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.472+0000 7f9cfe000640 1 --2- 172.21.15.131:0/2045484089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1a00 0x7f9d000b4b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.472+0000 7f9cfea00640 1 -- 172.21.15.131:0/2045484089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1a00 msgr2=0x7f9d000b4b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:37:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.472+0000 7f9cfea00640 1 --2- 172.21.15.131:0/2045484089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1a00 0x7f9d000b4b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.472+0000 7f9cfea00640 1 -- 172.21.15.131:0/2045484089 >> v1:172.21.15.131:6789/0 conn(0x7f9d000b50b0 legacy=0x7f9d000b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.472+0000 7f9cfd600640 1 -- 172.21.15.131:0/2045484089 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4241639937 0 0) 0x7f9d000d0480 con 0x7f9d000b50b0 2024-04-28T12:37:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.472+0000 7f9cfea00640 1 -- 172.21.15.131:0/2045484089 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d000d0a00 con 0x7f9d000d1620 2024-04-28T12:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.473+0000 7f9cfea00640 1 --2- 172.21.15.131:0/2045484089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d000d1620 0x7f9d000d5750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00097a0 tx=0x7f9cf002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b971e56c6355a246 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.473+0000 7f9cfd600640 1 -- 172.21.15.131:0/2045484089 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0037070 con 0x7f9d000d1620 2024-04-28T12:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.473+0000 7f9cfd600640 1 -- 172.21.15.131:0/2045484089 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf002ae50 con 0x7f9d000d1620 2024-04-28T12:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.473+0000 7f9cfd600640 1 -- 172.21.15.131:0/2045484089 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0032660 con 0x7f9d000d1620 2024-04-28T12:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.474+0000 7f9cffe00640 1 -- 172.21.15.131:0/2045484089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d000d1620 msgr2=0x7f9d000d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.474+0000 7f9cffe00640 1 --2- 172.21.15.131:0/2045484089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d000d1620 0x7f9d000d5750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00097a0 tx=0x7f9cf002a100 comp rx=0 tx=0).stop 2024-04-28T12:37:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.474+0000 7f9cffe00640 1 -- 172.21.15.131:0/2045484089 shutdown_connections 2024-04-28T12:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.474+0000 7f9cffe00640 1 --2- 172.21.15.131:0/2045484089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1a00 0x7f9d000b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.474+0000 7f9cffe00640 1 --2- 172.21.15.131:0/2045484089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d000d1620 0x7f9d000d5750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.474+0000 7f9cffe00640 1 -- 172.21.15.131:0/2045484089 >> 172.21.15.131:0/2045484089 conn(0x7f9d000d4c30 msgr2=0x7f9d000d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.474+0000 7f9cffe00640 1 -- 172.21.15.131:0/2045484089 shutdown_connections 2024-04-28T12:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.474+0000 7f9cffe00640 1 -- 172.21.15.131:0/2045484089 wait complete. 2024-04-28T12:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.476+0000 7f9cffe00640 1 Processor -- start 2024-04-28T12:37:06.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.476+0000 7f9cffe00640 1 -- start start 2024-04-28T12:37:06.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.477+0000 7f9cffe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d000b50b0 0x7f9d001f9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:06.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.477+0000 7f9cffe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1620 0x7f9d000cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:06.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.477+0000 7f9cffe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d000d1a00 0x7f9d001f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:06.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.477+0000 7f9cffe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9d000b5c50 con 0x7f9d000d1a00 2024-04-28T12:37:06.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.477+0000 7f9cffe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9d000d1e60 con 0x7f9d000b50b0 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.477+0000 7f9cffe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9d000b5dd0 con 0x7f9d000d1620 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.477+0000 7f9cfea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d000b50b0 0x7f9d001f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.477+0000 7f9cfea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d000b50b0 0x7f9d001f9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:54140/0 (socket says 172.21.15.131:54140) 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.477+0000 7f9cfea00640 1 -- 172.21.15.131:0/3821617212 learned_addr learned my addr 172.21.15.131:0/3821617212 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.477+0000 7f9cfe000640 1 --2- 172.21.15.131:0/3821617212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1620 0x7f9d000cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.477+0000 7f9cff400640 1 --2- 172.21.15.131:0/3821617212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d000d1a00 0x7f9d001f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cfe000640 1 -- 172.21.15.131:0/3821617212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1620 msgr2=0x7f9d000cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cfe000640 1 -- 172.21.15.131:0/3821617212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1620 msgr2=0x7f9d000cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cfe000640 1 --2- 172.21.15.131:0/3821617212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1620 0x7f9d000cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cfea00640 1 -- 172.21.15.131:0/3821617212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1620 msgr2=0x7f9d000cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cfe000640 1 --2- 172.21.15.131:0/3821617212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1620 0x7f9d000cd980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cfea00640 1 --2- 172.21.15.131:0/3821617212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1620 0x7f9d000cd980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cfea00640 1 -- 172.21.15.131:0/3821617212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d000d1a00 msgr2=0x7f9d001f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cfea00640 1 --2- 172.21.15.131:0/3821617212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d000d1a00 0x7f9d001f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cfea00640 1 -- 172.21.15.131:0/3821617212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d000cdec0 con 0x7f9d000b50b0 2024-04-28T12:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cfea00640 1 --2- 172.21.15.131:0/3821617212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d000b50b0 0x7f9d001f9410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00098c0 tx=0x7f9cf00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cf7400640 1 -- 172.21.15.131:0/3821617212 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0041020 con 0x7f9d000b50b0 2024-04-28T12:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cf7400640 1 -- 172.21.15.131:0/3821617212 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf003c670 con 0x7f9d000b50b0 2024-04-28T12:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.478+0000 7f9cffe00640 1 -- 172.21.15.131:0/3821617212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d000ce190 con 0x7f9d000b50b0 2024-04-28T12:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.479+0000 7f9cf7400640 1 -- 172.21.15.131:0/3821617212 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0037040 con 0x7f9d000b50b0 2024-04-28T12:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.479+0000 7f9cffe00640 1 -- 172.21.15.131:0/3821617212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d000ce750 con 0x7f9d000b50b0 2024-04-28T12:37:06.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.479+0000 7f9cf7400640 1 -- 172.21.15.131:0/3821617212 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9cf0044020 con 0x7f9d000b50b0 2024-04-28T12:37:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.479+0000 7f9cf7400640 1 -- 172.21.15.131:0/3821617212 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f9cf003ba90 con 0x7f9d000b50b0 2024-04-28T12:37:06.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.480+0000 7f9cffe00640 1 -- 172.21.15.131:0/3821617212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d00069350 con 0x7f9d000b50b0 2024-04-28T12:37:06.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.484+0000 7f9cf7400640 1 -- 172.21.15.131:0/3821617212 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9cf004b070 con 0x7f9d000b50b0 2024-04-28T12:37:06.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.547+0000 7f9cffe00640 1 -- 172.21.15.131:0/3821617212 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d0006d960 con 0x7f9d000b50b0 2024-04-28T12:37:06.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.548+0000 7f9cf7400640 1 -- 172.21.15.131:0/3821617212 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f9cf003bce0 con 0x7f9d000b50b0 2024-04-28T12:37:06.546 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:06.546 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":10,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:06.178042+0000","last_up_change":"0.000000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.550+0000 7f9cffe00640 1 -- 172.21.15.131:0/3821617212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d000b50b0 msgr2=0x7f9d001f9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.550+0000 7f9cffe00640 1 --2- 172.21.15.131:0/3821617212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d000b50b0 0x7f9d001f9410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00098c0 tx=0x7f9cf00046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.550+0000 7f9cffe00640 1 -- 172.21.15.131:0/3821617212 shutdown_connections 2024-04-28T12:37:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.550+0000 7f9cffe00640 1 --2- 172.21.15.131:0/3821617212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d000d1a00 0x7f9d001f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.550+0000 7f9cffe00640 1 --2- 172.21.15.131:0/3821617212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d000d1620 0x7f9d000cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.550+0000 7f9cffe00640 1 --2- 172.21.15.131:0/3821617212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d000b50b0 0x7f9d001f9410 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.550+0000 7f9cffe00640 1 -- 172.21.15.131:0/3821617212 >> 172.21.15.131:0/3821617212 conn(0x7f9d000d4c30 msgr2=0x7f9d0006db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.550+0000 7f9cffe00640 1 -- 172.21.15.131:0/3821617212 shutdown_connections 2024-04-28T12:37:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:06.550+0000 7f9cffe00640 1 -- 172.21.15.131:0/3821617212 wait complete. 2024-04-28T12:37:06.562 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-28T12:37:06.563 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-28T12:37:06.568 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:06.571+0000 7fb17f1bc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-28T12:37:06.625 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:06.628+0000 7fb17f1bc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-28T12:37:06.758 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:06.761+0000 7f51361ee040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-28T12:37:06.808 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:06.811+0000 7fb17f1bc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-28T12:37:06.889 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:06.892+0000 7f51361ee040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-28T12:37:06.908 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-28T12:37:06.911+0000 7fdb43e0c640 -1 Falling back to public interface 2024-04-28T12:37:06.911 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-04-28T12:37:06.915+0000 7f0679616640 -1 Falling back to public interface 2024-04-28T12:37:06.915 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-04-28T12:37:06.919+0000 7f1e6100d640 -1 Falling back to public interface 2024-04-28T12:37:06.916 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-04-28T12:37:06.920+0000 7fb179b4c640 -1 Falling back to public interface 2024-04-28T12:37:06.918 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-04-28T12:37:06.922+0000 7f2bd79a1640 -1 Falling back to public interface 2024-04-28T12:37:06.919 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-04-28T12:37:06.922+0000 7f8e94c1d640 -1 Falling back to public interface 2024-04-28T12:37:06.927 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-04-28T12:37:06.930+0000 7f8e93200640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6825/0: (98) Address already in use 2024-04-28T12:37:06.942 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-04-28T12:37:06.945+0000 7f9132b28640 -1 Falling back to public interface 2024-04-28T12:37:06.946 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-04-28T12:37:06.949+0000 7fe399b3b640 -1 Falling back to public interface 2024-04-28T12:37:06.983 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:06.986+0000 7fb17f1bc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-28T12:37:07.041 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:07.044+0000 7fb17f1bc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-28T12:37:07.121 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:07.124+0000 7fb17f1bc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-28T12:37:07.173 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:07.176+0000 7fb17f1bc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-28T12:37:07.368 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:07.524 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.527+0000 7f1f67e00640 1 Processor -- start 2024-04-28T12:37:07.524 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.527+0000 7f1f67e00640 1 -- start start 2024-04-28T12:37:07.526 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.528+0000 7f1f67e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f6815efe0 0x7f1f680736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.528+0000 7f1f67e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1f680d4430 0x7f1f68072940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.528+0000 7f1f67e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f68073c20 con 0x7f1f68072e80 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.528+0000 7f1f67e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1f680d3fa0 con 0x7f1f6815efe0 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.528+0000 7f1f67e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1f680ba5f0 con 0x7f1f680d4430 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.528+0000 7f1f66a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f6815efe0 0x7f1f680736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.528+0000 7f1f66a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f6815efe0 0x7f1f680736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45486/0 (socket says 172.21.15.131:45486) 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.528+0000 7f1f66a00640 1 -- 172.21.15.131:0/1288069279 learned_addr learned my addr 172.21.15.131:0/1288069279 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.529+0000 7f1f66000640 1 --2- 172.21.15.131:0/1288069279 >> v2:172.21.15.157:3300/0 conn(0x7f1f680d4430 0x7f1f68072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.529+0000 7f1f66a00640 1 -- 172.21.15.131:0/1288069279 >> v2:172.21.15.157:3300/0 conn(0x7f1f680d4430 msgr2=0x7f1f68072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.529+0000 7f1f66a00640 1 --2- 172.21.15.131:0/1288069279 >> v2:172.21.15.157:3300/0 conn(0x7f1f680d4430 0x7f1f68072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.529+0000 7f1f66a00640 1 -- 172.21.15.131:0/1288069279 >> v1:172.21.15.157:6789/0 conn(0x7f1f68072e80 legacy=0x7f1f680d28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.529+0000 7f1f66a00640 1 -- 172.21.15.131:0/1288069279 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f680ba8d0 con 0x7f1f6815efe0 2024-04-28T12:37:07.527 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.529+0000 7f1f66000640 1 --2- 172.21.15.131:0/1288069279 >> v2:172.21.15.157:3300/0 conn(0x7f1f680d4430 0x7f1f68072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:07.528 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.530+0000 7f1f66a00640 1 --2- 172.21.15.131:0/1288069279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f6815efe0 0x7f1f680736e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1f6808d000 tx=0x7f1f5802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6cc9eb2ec8ff0e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:07.529 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.531+0000 7f1f65600640 1 -- 172.21.15.131:0/1288069279 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58037070 con 0x7f1f6815efe0 2024-04-28T12:37:07.529 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.531+0000 7f1f65600640 1 -- 172.21.15.131:0/1288069279 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f58032440 con 0x7f1f6815efe0 2024-04-28T12:37:07.529 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.531+0000 7f1f65600640 1 -- 172.21.15.131:0/1288069279 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58032810 con 0x7f1f6815efe0 2024-04-28T12:37:07.529 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.531+0000 7f1f67e00640 1 -- 172.21.15.131:0/1288069279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f6815efe0 msgr2=0x7f1f680736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:07.529 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.531+0000 7f1f67e00640 1 --2- 172.21.15.131:0/1288069279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f6815efe0 0x7f1f680736e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1f6808d000 tx=0x7f1f5802a230 comp rx=0 tx=0).stop 2024-04-28T12:37:07.529 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.531+0000 7f1f67e00640 1 -- 172.21.15.131:0/1288069279 shutdown_connections 2024-04-28T12:37:07.529 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.531+0000 7f1f67e00640 1 --2- 172.21.15.131:0/1288069279 >> v2:172.21.15.157:3300/0 conn(0x7f1f680d4430 0x7f1f68072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:07.529 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.531+0000 7f1f67e00640 1 --2- 172.21.15.131:0/1288069279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f6815efe0 0x7f1f680736e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:07.529 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.531+0000 7f1f67e00640 1 -- 172.21.15.131:0/1288069279 >> 172.21.15.131:0/1288069279 conn(0x7f1f680d4cd0 msgr2=0x7f1f680d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:07.529 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.531+0000 7f1f67e00640 1 -- 172.21.15.131:0/1288069279 shutdown_connections 2024-04-28T12:37:07.529 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.531+0000 7f1f67e00640 1 -- 172.21.15.131:0/1288069279 wait complete. 2024-04-28T12:37:07.530 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.533+0000 7f1f67e00640 1 Processor -- start 2024-04-28T12:37:07.530 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.533+0000 7f1f67e00640 1 -- start start 2024-04-28T12:37:07.533 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.534+0000 7f1f67e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f68072e80 0x7f1f6815b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:07.533 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.534+0000 7f1f67e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f680d4430 0x7f1f680b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:07.533 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.534+0000 7f1f67e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6815efe0 0x7f1f680b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:07.534 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.534+0000 7f1f67e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6815f3c0 con 0x7f1f680d4430 2024-04-28T12:37:07.534 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.534+0000 7f1f67e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1f680d3c30 con 0x7f1f6815efe0 2024-04-28T12:37:07.534 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.534+0000 7f1f67e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1f680babb0 con 0x7f1f68072e80 2024-04-28T12:37:07.534 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.534+0000 7f1f66000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f680d4430 0x7f1f680b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:07.534 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f66000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f680d4430 0x7f1f680b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45490/0 (socket says 172.21.15.131:45490) 2024-04-28T12:37:07.534 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f66000640 1 -- 172.21.15.131:0/2794956801 learned_addr learned my addr 172.21.15.131:0/2794956801 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:07.534 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f66a00640 1 --2- 172.21.15.131:0/2794956801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f68072e80 0x7f1f6815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:07.534 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f67400640 1 --2- 172.21.15.131:0/2794956801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6815efe0 0x7f1f680b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:07.534 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f66000640 1 -- 172.21.15.131:0/2794956801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f68072e80 msgr2=0x7f1f6815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:07.535 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f66000640 1 --2- 172.21.15.131:0/2794956801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f68072e80 0x7f1f6815b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:07.535 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f66000640 1 -- 172.21.15.131:0/2794956801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6815efe0 msgr2=0x7f1f680b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:07.535 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f66000640 1 --2- 172.21.15.131:0/2794956801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6815efe0 0x7f1f680b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:07.535 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f66000640 1 -- 172.21.15.131:0/2794956801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f681f56a0 con 0x7f1f680d4430 2024-04-28T12:37:07.535 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f66000640 1 --2- 172.21.15.131:0/2794956801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f680d4430 0x7f1f680b5d20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400c7f0 tx=0x7f1f5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:07.535 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f5ea00640 1 -- 172.21.15.131:0/2794956801 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54015070 con 0x7f1f680d4430 2024-04-28T12:37:07.535 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f5ea00640 1 -- 172.21.15.131:0/2794956801 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f54007820 con 0x7f1f680d4430 2024-04-28T12:37:07.535 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f67e00640 1 -- 172.21.15.131:0/2794956801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f681f5c40 con 0x7f1f680d4430 2024-04-28T12:37:07.535 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.535+0000 7f1f5ea00640 1 -- 172.21.15.131:0/2794956801 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54007c10 con 0x7f1f680d4430 2024-04-28T12:37:07.535 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.536+0000 7f1f67e00640 1 -- 172.21.15.131:0/2794956801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6815b680 con 0x7f1f680d4430 2024-04-28T12:37:07.535 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.537+0000 7f1f67e00640 1 -- 172.21.15.131:0/2794956801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f680506c0 con 0x7f1f680d4430 2024-04-28T12:37:07.536 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.537+0000 7f1f5ea00640 1 -- 172.21.15.131:0/2794956801 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 2) ==== 45545+0+0 (secure 0 0 0) 0x7f1f54018070 con 0x7f1f680d4430 2024-04-28T12:37:07.536 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.538+0000 7f1f5ea00640 1 -- 172.21.15.131:0/2794956801 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f1f5404abc0 con 0x7f1f680d4430 2024-04-28T12:37:07.536 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.540+0000 7f1f5ea00640 1 -- 172.21.15.131:0/2794956801 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1f54010350 con 0x7f1f680d4430 2024-04-28T12:37:07.600 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.604+0000 7f1f67e00640 1 -- 172.21.15.131:0/2794956801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f6815e5c0 con 0x7f1f680d4430 2024-04-28T12:37:07.601 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.605+0000 7f1f5ea00640 1 -- 172.21.15.131:0/2794956801 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f1f54022da0 con 0x7f1f680d4430 2024-04-28T12:37:07.601 INFO:teuthology.misc.health.smithi131.stdout: 2024-04-28T12:37:07.601 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":10,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:06.178042+0000","last_up_change":"0.000000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:07.605 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.606+0000 7f1f67e00640 1 -- 172.21.15.131:0/2794956801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f680d4430 msgr2=0x7f1f680b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:07.605 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.606+0000 7f1f67e00640 1 --2- 172.21.15.131:0/2794956801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f680d4430 0x7f1f680b5d20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400c7f0 tx=0x7f1f5400ccb0 comp rx=0 tx=0).stop 2024-04-28T12:37:07.606 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.606+0000 7f1f67e00640 1 -- 172.21.15.131:0/2794956801 shutdown_connections 2024-04-28T12:37:07.606 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.606+0000 7f1f67e00640 1 --2- 172.21.15.131:0/2794956801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f6815efe0 0x7f1f680b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:07.606 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.606+0000 7f1f67e00640 1 --2- 172.21.15.131:0/2794956801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f680d4430 0x7f1f680b5d20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:07.606 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.606+0000 7f1f67e00640 1 --2- 172.21.15.131:0/2794956801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f68072e80 0x7f1f6815b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:07.606 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.606+0000 7f1f67e00640 1 -- 172.21.15.131:0/2794956801 >> 172.21.15.131:0/2794956801 conn(0x7f1f680d4cd0 msgr2=0x7f1f680cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:07.606 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.606+0000 7f1f67e00640 1 -- 172.21.15.131:0/2794956801 shutdown_connections 2024-04-28T12:37:07.606 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:07.606+0000 7f1f67e00640 1 -- 172.21.15.131:0/2794956801 wait complete. 2024-04-28T12:37:07.616 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-28T12:37:07.720 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:07.723+0000 7fb17f1bc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-28T12:37:07.853 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-04-28T12:37:07.857+0000 7fb17f1bc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-28T12:37:07.938 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:07.941+0000 7f50d6800640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-28T12:37:07.939 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-04-28T12:37:07.942+0000 7f50d6800640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-28T12:37:10.252 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-04-28T12:37:10.256+0000 7f0679616640 -1 osd.1 0 log_to_monitors true 2024-04-28T12:37:10.305 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-04-28T12:37:10.308+0000 7fb179b4c640 -1 osd.6 0 log_to_monitors true 2024-04-28T12:37:10.560 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-04-28T12:37:10.563+0000 7f1e6100d640 -1 osd.2 0 log_to_monitors true 2024-04-28T12:37:10.561 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-04-28T12:37:10.565+0000 7f2bd79a1640 -1 osd.0 0 log_to_monitors true 2024-04-28T12:37:10.604 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-04-28T12:37:10.607+0000 7f9132b28640 -1 osd.5 0 log_to_monitors true 2024-04-28T12:37:10.610 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-04-28T12:37:10.613+0000 7fe399b3b640 -1 osd.7 0 log_to_monitors true 2024-04-28T12:37:10.726 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-28T12:37:10.729+0000 7fdb43e0c640 -1 osd.4 0 log_to_monitors true 2024-04-28T12:37:10.765 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-04-28T12:37:10.769+0000 7f8e94c1d640 -1 osd.3 0 log_to_monitors true 2024-04-28T12:37:13.651 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-04-28T12:37:13.653+0000 7fe395400640 -1 osd.7 0 waiting for initial osdmap 2024-04-28T12:37:13.657 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-04-28T12:37:13.661+0000 7fe38aa00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:37:13.746 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-04-28T12:37:13.748+0000 7fb174a00640 -1 osd.6 0 waiting for initial osdmap 2024-04-28T12:37:13.751 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-28T12:37:13.754+0000 7fdb3f000640 -1 osd.4 0 waiting for initial osdmap 2024-04-28T12:37:13.757 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-04-28T12:37:13.761+0000 7fb16aa00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:37:13.765 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-04-28T12:37:13.768+0000 7fdb34600640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:37:13.781 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-04-28T12:37:13.784+0000 7f912e400640 -1 osd.5 0 waiting for initial osdmap 2024-04-28T12:37:13.787 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-04-28T12:37:13.790+0000 7f9123a00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:37:14.299 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-04-28T12:37:14.302+0000 7f0674800640 -1 osd.1 0 waiting for initial osdmap 2024-04-28T12:37:14.314 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-04-28T12:37:14.317+0000 7f0669e00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:37:14.318 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-04-28T12:37:14.320+0000 7f2bd2800640 -1 osd.0 0 waiting for initial osdmap 2024-04-28T12:37:14.325 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-04-28T12:37:14.328+0000 7f1e5ba00640 -1 osd.2 0 waiting for initial osdmap 2024-04-28T12:37:14.328 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-04-28T12:37:14.332+0000 7f8e90000640 -1 osd.3 0 waiting for initial osdmap 2024-04-28T12:37:14.331 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-04-28T12:37:14.334+0000 7f2bc8800640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:37:14.334 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-04-28T12:37:14.337+0000 7f1e51a00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:37:14.343 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-04-28T12:37:14.347+0000 7f8e85600640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:37:14.423 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:14.581 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.584+0000 7fb33a000640 1 Processor -- start 2024-04-28T12:37:14.581 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.584+0000 7fb33a000640 1 -- start start 2024-04-28T12:37:14.582 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.585+0000 7fb33a000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb33415f1a0 0x7fb334162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:14.582 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.585+0000 7fb33a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3340babe0 0x7fb3340cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:14.582 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.585+0000 7fb33a000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb334163750 con 0x7fb334163240 2024-04-28T12:37:14.582 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.586+0000 7fb33a000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb3340d2f80 con 0x7fb33415f1a0 2024-04-28T12:37:14.582 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.586+0000 7fb33a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb334163a30 con 0x7fb3340babe0 2024-04-28T12:37:14.583 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.586+0000 7fb332a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3340babe0 0x7fb3340cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:14.583 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.586+0000 7fb333400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb33415f1a0 0x7fb334162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:14.583 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.586+0000 7fb333400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb33415f1a0 0x7fb334162d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60650/0 (socket says 172.21.15.131:60650) 2024-04-28T12:37:14.583 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.586+0000 7fb333400640 1 -- 172.21.15.131:0/1166984008 learned_addr learned my addr 172.21.15.131:0/1166984008 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:14.583 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.586+0000 7fb332a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3340babe0 0x7fb3340cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:54076/0 (socket says 172.21.15.131:54076) 2024-04-28T12:37:14.584 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.587+0000 7fb332a00640 1 -- 172.21.15.131:0/1166984008 >> v2:172.21.15.157:3300/0 conn(0x7fb33415f1a0 msgr2=0x7fb334162d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:14.584 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.587+0000 7fb332a00640 1 --2- 172.21.15.131:0/1166984008 >> v2:172.21.15.157:3300/0 conn(0x7fb33415f1a0 0x7fb334162d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:14.584 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.587+0000 7fb332a00640 1 -- 172.21.15.131:0/1166984008 >> v1:172.21.15.157:6789/0 conn(0x7fb334163240 legacy=0x7fb3340b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:14.584 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.587+0000 7fb332a00640 1 -- 172.21.15.131:0/1166984008 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3340d31d0 con 0x7fb3340babe0 2024-04-28T12:37:14.584 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.587+0000 7fb333400640 1 --2- 172.21.15.131:0/1166984008 >> v2:172.21.15.157:3300/0 conn(0x7fb33415f1a0 0x7fb334162d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:37:14.586 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.589+0000 7fb332a00640 1 --2- 172.21.15.131:0/1166984008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3340babe0 0x7fb3340cfbf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb3240097a0 tx=0x7fb32402a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30509675b44a5352 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:14.586 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.590+0000 7fb332000640 1 -- 172.21.15.131:0/1166984008 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324037070 con 0x7fb3340babe0 2024-04-28T12:37:14.587 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.590+0000 7fb332000640 1 -- 172.21.15.131:0/1166984008 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb32402ac90 con 0x7fb3340babe0 2024-04-28T12:37:14.587 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.590+0000 7fb332000640 1 -- 172.21.15.131:0/1166984008 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324032660 con 0x7fb3340babe0 2024-04-28T12:37:14.587 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.590+0000 7fb33a000640 1 -- 172.21.15.131:0/1166984008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3340babe0 msgr2=0x7fb3340cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:14.588 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.590+0000 7fb33a000640 1 --2- 172.21.15.131:0/1166984008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3340babe0 0x7fb3340cfbf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb3240097a0 tx=0x7fb32402a000 comp rx=0 tx=0).stop 2024-04-28T12:37:14.588 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.591+0000 7fb33a000640 1 -- 172.21.15.131:0/1166984008 shutdown_connections 2024-04-28T12:37:14.588 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.591+0000 7fb33a000640 1 --2- 172.21.15.131:0/1166984008 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3340babe0 0x7fb3340cfbf0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:14.588 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.591+0000 7fb33a000640 1 --2- 172.21.15.131:0/1166984008 >> v2:172.21.15.157:3300/0 conn(0x7fb33415f1a0 0x7fb334162d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:14.588 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.591+0000 7fb33a000640 1 -- 172.21.15.131:0/1166984008 >> 172.21.15.131:0/1166984008 conn(0x7fb334073b60 msgr2=0x7fb334073f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:14.588 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.591+0000 7fb33a000640 1 -- 172.21.15.131:0/1166984008 shutdown_connections 2024-04-28T12:37:14.588 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.591+0000 7fb33a000640 1 -- 172.21.15.131:0/1166984008 wait complete. 2024-04-28T12:37:14.589 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.592+0000 7fb33a000640 1 Processor -- start 2024-04-28T12:37:14.589 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.592+0000 7fb33a000640 1 -- start start 2024-04-28T12:37:14.590 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.593+0000 7fb33a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3340babe0 0x7fb33415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:14.590 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.593+0000 7fb33a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb33415f1a0 0x7fb3340c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:14.590 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.593+0000 7fb33a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb334163240 0x7fb334143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:14.590 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.593+0000 7fb33a000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb3340d49e0 con 0x7fb33415f1a0 2024-04-28T12:37:14.590 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.593+0000 7fb33a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb33415f580 con 0x7fb334163240 2024-04-28T12:37:14.590 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.593+0000 7fb33a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb3340d4b60 con 0x7fb3340babe0 2024-04-28T12:37:14.590 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.593+0000 7fb332a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb33415f1a0 0x7fb3340c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:14.590 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.593+0000 7fb332a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb33415f1a0 0x7fb3340c7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59862/0 (socket says 172.21.15.131:59862) 2024-04-28T12:37:14.590 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.593+0000 7fb332a00640 1 -- 172.21.15.131:0/3924868470 learned_addr learned my addr 172.21.15.131:0/3924868470 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:14.591 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.594+0000 7fb332a00640 1 -- 172.21.15.131:0/3924868470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3340babe0 msgr2=0x7fb33415aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:14.591 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.594+0000 7fb332a00640 1 --2- 172.21.15.131:0/3924868470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3340babe0 0x7fb33415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:14.591 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.594+0000 7fb332a00640 1 -- 172.21.15.131:0/3924868470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb334163240 msgr2=0x7fb334143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:14.592 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.594+0000 7fb332a00640 1 --2- 172.21.15.131:0/3924868470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb334163240 0x7fb334143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:14.592 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.594+0000 7fb332a00640 1 -- 172.21.15.131:0/3924868470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb334141a40 con 0x7fb33415f1a0 2024-04-28T12:37:14.592 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.594+0000 7fb332a00640 1 --2- 172.21.15.131:0/3924868470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb33415f1a0 0x7fb3340c7970 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb324035040 tx=0x7fb3240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:14.592 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.594+0000 7fb32b400640 1 -- 172.21.15.131:0/3924868470 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324041030 con 0x7fb33415f1a0 2024-04-28T12:37:14.592 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.594+0000 7fb32b400640 1 -- 172.21.15.131:0/3924868470 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb32402aa90 con 0x7fb33415f1a0 2024-04-28T12:37:14.592 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.594+0000 7fb32b400640 1 -- 172.21.15.131:0/3924868470 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324037040 con 0x7fb33415f1a0 2024-04-28T12:37:14.592 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.594+0000 7fb33a000640 1 -- 172.21.15.131:0/3924868470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb334141d20 con 0x7fb33415f1a0 2024-04-28T12:37:14.592 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.594+0000 7fb33a000640 1 -- 172.21.15.131:0/3924868470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3341422e0 con 0x7fb33415f1a0 2024-04-28T12:37:14.593 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.595+0000 7fb33a000640 1 -- 172.21.15.131:0/3924868470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb334069c80 con 0x7fb33415f1a0 2024-04-28T12:37:14.593 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.596+0000 7fb32b400640 1 -- 172.21.15.131:0/3924868470 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fb324044070 con 0x7fb33415f1a0 2024-04-28T12:37:14.593 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.597+0000 7fb32b400640 1 --2- 172.21.15.131:0/3924868470 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb31006c860 0x7fb31006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:14.594 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.597+0000 7fb32b400640 1 -- 172.21.15.131:0/3924868470 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fb3240ab1c0 con 0x7fb33415f1a0 2024-04-28T12:37:14.594 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.598+0000 7fb333400640 1 --2- 172.21.15.131:0/3924868470 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb31006c860 0x7fb31006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:14.596 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.599+0000 7fb333400640 1 --2- 172.21.15.131:0/3924868470 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb31006c860 0x7fb31006ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009630 tx=0x7fb31c006e90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:14.599 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.602+0000 7fb32b400640 1 -- 172.21.15.131:0/3924868470 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb32407a570 con 0x7fb33415f1a0 2024-04-28T12:37:14.803 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.806+0000 7fb33a000640 1 -- 172.21.15.131:0/3924868470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb334162da0 con 0x7fb33415f1a0 2024-04-28T12:37:14.804 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.807+0000 7fb32b400640 1 -- 172.21.15.131:0/3924868470 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10933 (secure 0 0 0) 0x7fb324079e70 con 0x7fb33415f1a0 2024-04-28T12:37:14.804 INFO:teuthology.misc.health.smithi131.stdout: 2024-04-28T12:37:14.804 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":13,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:14.661042+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:14.807 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.810+0000 7fb33a000640 1 -- 172.21.15.131:0/3924868470 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb31006c860 msgr2=0x7fb31006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:14.807 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.810+0000 7fb33a000640 1 --2- 172.21.15.131:0/3924868470 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb31006c860 0x7fb31006ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009630 tx=0x7fb31c006e90 comp rx=0 tx=0).stop 2024-04-28T12:37:14.807 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.810+0000 7fb33a000640 1 -- 172.21.15.131:0/3924868470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb33415f1a0 msgr2=0x7fb3340c7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:14.808 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.810+0000 7fb33a000640 1 --2- 172.21.15.131:0/3924868470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb33415f1a0 0x7fb3340c7970 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb324035040 tx=0x7fb3240046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:14.808 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.810+0000 7fb33a000640 1 -- 172.21.15.131:0/3924868470 shutdown_connections 2024-04-28T12:37:14.808 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.810+0000 7fb33a000640 1 --2- 172.21.15.131:0/3924868470 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb31006c860 0x7fb31006ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:14.808 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.810+0000 7fb33a000640 1 --2- 172.21.15.131:0/3924868470 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb334163240 0x7fb334143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:14.808 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.810+0000 7fb33a000640 1 --2- 172.21.15.131:0/3924868470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb33415f1a0 0x7fb3340c7970 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:14.808 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.810+0000 7fb33a000640 1 --2- 172.21.15.131:0/3924868470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3340babe0 0x7fb33415aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:14.808 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.810+0000 7fb33a000640 1 -- 172.21.15.131:0/3924868470 >> 172.21.15.131:0/3924868470 conn(0x7fb334073b60 msgr2=0x7fb3340b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:14.808 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.810+0000 7fb33a000640 1 -- 172.21.15.131:0/3924868470 shutdown_connections 2024-04-28T12:37:14.808 INFO:teuthology.misc.health.smithi131.stderr:2024-04-28T12:37:14.810+0000 7fb33a000640 1 -- 172.21.15.131:0/3924868470 wait complete. 2024-04-28T12:37:14.821 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-28T12:37:14.821 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-28T12:37:14.822 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-28T12:37:14.822 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-28T12:37:14.826 INFO:tasks.ceph.mds.a:Started 2024-04-28T12:37:14.827 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-28T12:37:14.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-28T12:37:14.831 INFO:tasks.ceph.mds.c:Started 2024-04-28T12:37:14.831 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-28T12:37:14.831 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-28T12:37:14.839 INFO:tasks.ceph.mds.b:Started 2024-04-28T12:37:14.840 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-28T12:37:14.840 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-28T12:37:14.848 INFO:tasks.ceph.mds.d:Started 2024-04-28T12:37:14.848 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-28T12:37:14.849 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:14.965 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-04-28T12:37:14.965 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-04-28T12:37:14.987 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-28T12:37:14.996 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-28T12:37:15.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.025+0000 7efc5a400640 1 Processor -- start 2024-04-28T12:37:15.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.025+0000 7efc5a400640 1 -- start start 2024-04-28T12:37:15.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.026+0000 7efc5a400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efc540b71a0 0x7efc540d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:15.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.027+0000 7efc5a400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5415f0d0 0x7efc540cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:15.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.027+0000 7efc5a400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc540d3a10 con 0x7efc540b7580 2024-04-28T12:37:15.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.027+0000 7efc5a400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7efc540c8830 con 0x7efc540b71a0 2024-04-28T12:37:15.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.027+0000 7efc5a400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc540d3cb0 con 0x7efc5415f0d0 2024-04-28T12:37:15.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.027+0000 7efc53400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efc540b71a0 0x7efc540d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:15.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.027+0000 7efc53400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efc540b71a0 0x7efc540d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59896/0 (socket says 172.21.15.131:59896) 2024-04-28T12:37:15.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.027+0000 7efc53400640 1 -- 172.21.15.131:0/2215201336 learned_addr learned my addr 172.21.15.131:0/2215201336 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:15.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.027+0000 7efc53e00640 1 --2- 172.21.15.131:0/2215201336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5415f0d0 0x7efc540cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:15.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.027+0000 7efc53400640 1 -- 172.21.15.131:0/2215201336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5415f0d0 msgr2=0x7efc540cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:15.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.027+0000 7efc53400640 1 --2- 172.21.15.131:0/2215201336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5415f0d0 0x7efc540cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.027+0000 7efc53400640 1 -- 172.21.15.131:0/2215201336 >> v1:172.21.15.157:6789/0 conn(0x7efc540b7580 legacy=0x7efc5415e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:15.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.027+0000 7efc53400640 1 -- 172.21.15.131:0/2215201336 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc540c8af0 con 0x7efc540b71a0 2024-04-28T12:37:15.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.028+0000 7efc53400640 1 --2- 172.21.15.131:0/2215201336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc540b71a0 0x7efc540d34d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efc44009870 tx=0x7efc4402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bb20dad7ead62 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:15.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.028+0000 7efc52000640 1 -- 172.21.15.131:0/2215201336 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44037070 con 0x7efc540b71a0 2024-04-28T12:37:15.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.028+0000 7efc52000640 1 -- 172.21.15.131:0/2215201336 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc44032440 con 0x7efc540b71a0 2024-04-28T12:37:15.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.028+0000 7efc52000640 1 -- 172.21.15.131:0/2215201336 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44032810 con 0x7efc540b71a0 2024-04-28T12:37:15.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.029+0000 7efc5a400640 1 -- 172.21.15.131:0/2215201336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc540b71a0 msgr2=0x7efc540d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:15.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.029+0000 7efc5a400640 1 --2- 172.21.15.131:0/2215201336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc540b71a0 0x7efc540d34d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efc44009870 tx=0x7efc4402a1c0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.029+0000 7efc5a400640 1 -- 172.21.15.131:0/2215201336 shutdown_connections 2024-04-28T12:37:15.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.029+0000 7efc5a400640 1 --2- 172.21.15.131:0/2215201336 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5415f0d0 0x7efc540cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.029+0000 7efc5a400640 1 --2- 172.21.15.131:0/2215201336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc540b71a0 0x7efc540d34d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.029+0000 7efc5a400640 1 -- 172.21.15.131:0/2215201336 >> 172.21.15.131:0/2215201336 conn(0x7efc540d4d70 msgr2=0x7efc540d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:15.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.029+0000 7efc5a400640 1 -- 172.21.15.131:0/2215201336 shutdown_connections 2024-04-28T12:37:15.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.029+0000 7efc5a400640 1 -- 172.21.15.131:0/2215201336 wait complete. 2024-04-28T12:37:15.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.031+0000 7efc5a400640 1 Processor -- start 2024-04-28T12:37:15.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.031+0000 7efc5a400640 1 -- start start 2024-04-28T12:37:15.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.032+0000 7efc5a400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc540b71a0 0x7efc5414f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:15.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.032+0000 7efc5a400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc540b7580 0x7efc540ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:15.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.032+0000 7efc5a400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5415f0d0 0x7efc540b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:15.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.032+0000 7efc5a400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efc5415f540 con 0x7efc540b7580 2024-04-28T12:37:15.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.032+0000 7efc5a400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc540c90b0 con 0x7efc540b71a0 2024-04-28T12:37:15.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.032+0000 7efc5a400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc5415f6c0 con 0x7efc5415f0d0 2024-04-28T12:37:15.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.032+0000 7efc52a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc540b7580 0x7efc540ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:15.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.032+0000 7efc52a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc540b7580 0x7efc540ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59912/0 (socket says 172.21.15.131:59912) 2024-04-28T12:37:15.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.032+0000 7efc53400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc540b71a0 0x7efc5414f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:15.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.032+0000 7efc52a00640 1 -- 172.21.15.131:0/1988378321 learned_addr learned my addr 172.21.15.131:0/1988378321 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:15.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.032+0000 7efc53e00640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5415f0d0 0x7efc540b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:15.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.033+0000 7efc53400640 1 -- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5415f0d0 msgr2=0x7efc540b9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:15.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.033+0000 7efc53400640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5415f0d0 0x7efc540b9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.033+0000 7efc53400640 1 -- 172.21.15.131:0/1988378321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc540b7580 msgr2=0x7efc540ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:15.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.033+0000 7efc53400640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc540b7580 0x7efc540ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.033+0000 7efc53400640 1 -- 172.21.15.131:0/1988378321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc540ccc30 con 0x7efc540b71a0 2024-04-28T12:37:15.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.033+0000 7efc53400640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc540b71a0 0x7efc5414f2b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efc44031000 tx=0x7efc440046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:15.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.033+0000 7efc4b400640 1 -- 172.21.15.131:0/1988378321 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44041020 con 0x7efc540b71a0 2024-04-28T12:37:15.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.033+0000 7efc4b400640 1 -- 172.21.15.131:0/1988378321 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc44030660 con 0x7efc540b71a0 2024-04-28T12:37:15.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.033+0000 7efc5a400640 1 -- 172.21.15.131:0/1988378321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc540ccf10 con 0x7efc540b71a0 2024-04-28T12:37:15.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.033+0000 7efc4b400640 1 -- 172.21.15.131:0/1988378321 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44037040 con 0x7efc540b71a0 2024-04-28T12:37:15.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.033+0000 7efc5a400640 1 -- 172.21.15.131:0/1988378321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc540cd4d0 con 0x7efc540b71a0 2024-04-28T12:37:15.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.034+0000 7efc5a400640 1 -- 172.21.15.131:0/1988378321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc54069350 con 0x7efc540b71a0 2024-04-28T12:37:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.035+0000 7efc4b400640 1 -- 172.21.15.131:0/1988378321 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7efc44044020 con 0x7efc540b71a0 2024-04-28T12:37:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.036+0000 7efc4b400640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efc3406c860 0x7efc3406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.036+0000 7efc4b400640 1 -- 172.21.15.131:0/1988378321 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7efc440abbe0 con 0x7efc540b71a0 2024-04-28T12:37:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.036+0000 7efc52a00640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efc3406c860 0x7efc3406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.037+0000 7efc52a00640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efc3406c860 0x7efc3406ed20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc5415c070 tx=0x7efc40009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.042+0000 7efc4b400640 1 -- 172.21.15.131:0/1988378321 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc4404b030 con 0x7efc540b71a0 2024-04-28T12:37:15.219 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:37:15.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.230+0000 7efc5a400640 1 -- 172.21.15.131:0/1988378321 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc5408ee00 con 0x7efc540b71a0 2024-04-28T12:37:15.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.232+0000 7efc4b400640 1 -- 172.21.15.131:0/1988378321 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10933 (secure 0 0 0) 0x7efc44079ed0 con 0x7efc540b71a0 2024-04-28T12:37:15.246 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:15.246 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":13,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:14.661042+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:15.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.234+0000 7efc5a400640 1 -- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efc3406c860 msgr2=0x7efc3406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:15.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.234+0000 7efc5a400640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efc3406c860 0x7efc3406ed20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc5415c070 tx=0x7efc40009040 comp rx=0 tx=0).stop 2024-04-28T12:37:15.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.234+0000 7efc5a400640 1 -- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc540b71a0 msgr2=0x7efc5414f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:15.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.234+0000 7efc5a400640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc540b71a0 0x7efc5414f2b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efc44031000 tx=0x7efc440046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.235+0000 7efc5a400640 1 -- 172.21.15.131:0/1988378321 shutdown_connections 2024-04-28T12:37:15.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.235+0000 7efc5a400640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efc3406c860 0x7efc3406ed20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.235+0000 7efc5a400640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5415f0d0 0x7efc540b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.235+0000 7efc5a400640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc540b7580 0x7efc540ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.235+0000 7efc5a400640 1 --2- 172.21.15.131:0/1988378321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc540b71a0 0x7efc5414f2b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.235+0000 7efc5a400640 1 -- 172.21.15.131:0/1988378321 >> 172.21.15.131:0/1988378321 conn(0x7efc540d4d70 msgr2=0x7efc540c9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:15.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.235+0000 7efc5a400640 1 -- 172.21.15.131:0/1988378321 shutdown_connections 2024-04-28T12:37:15.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.235+0000 7efc5a400640 1 -- 172.21.15.131:0/1988378321 wait complete. 2024-04-28T12:37:15.249 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-28T12:37:15.249 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:37:15.252 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:37:15.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.474+0000 7fbb82000640 1 Processor -- start 2024-04-28T12:37:15.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.474+0000 7fbb82000640 1 -- start start 2024-04-28T12:37:15.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.475+0000 7fbb82000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbb7c0cd180 0x7fbb7c072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:15.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.475+0000 7fbb82000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbb7c0ce480 0x7fbb7c06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.475+0000 7fbb82000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb7c0cd560 con 0x7fbb7c15dc50 2024-04-28T12:37:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.475+0000 7fbb82000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fbb7c072d00 con 0x7fbb7c0ce480 2024-04-28T12:37:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.475+0000 7fbb82000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbb7c072fe0 con 0x7fbb7c0cd180 2024-04-28T12:37:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.476+0000 7fbb81600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbb7c0ce480 0x7fbb7c06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.476+0000 7fbb81600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbb7c0ce480 0x7fbb7c06dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59922/0 (socket says 172.21.15.131:59922) 2024-04-28T12:37:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.476+0000 7fbb81600640 1 -- 172.21.15.131:0/402444022 learned_addr learned my addr 172.21.15.131:0/402444022 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.476+0000 7fbb81600640 1 -- 172.21.15.131:0/402444022 >> v2:172.21.15.157:3300/0 conn(0x7fbb7c0cd180 msgr2=0x7fbb7c072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.476+0000 7fbb81600640 1 --2- 172.21.15.131:0/402444022 >> v2:172.21.15.157:3300/0 conn(0x7fbb7c0cd180 0x7fbb7c072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.476+0000 7fbb81600640 1 -- 172.21.15.131:0/402444022 >> v1:172.21.15.157:6789/0 conn(0x7fbb7c15dc50 legacy=0x7fbb7c0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:15.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.476+0000 7fbb81600640 1 -- 172.21.15.131:0/402444022 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c0732c0 con 0x7fbb7c0ce480 2024-04-28T12:37:15.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.477+0000 7fbb81600640 1 --2- 172.21.15.131:0/402444022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c0ce480 0x7fbb7c06dac0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0717f0 tx=0x7fbb7402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdffb5c0c601105 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:15.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.477+0000 7fbb7be00640 1 -- 172.21.15.131:0/402444022 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74036070 con 0x7fbb7c0ce480 2024-04-28T12:37:15.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.477+0000 7fbb7be00640 1 -- 172.21.15.131:0/402444022 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb74031600 con 0x7fbb7c0ce480 2024-04-28T12:37:15.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.477+0000 7fbb7be00640 1 -- 172.21.15.131:0/402444022 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb740319d0 con 0x7fbb7c0ce480 2024-04-28T12:37:15.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.478+0000 7fbb82000640 1 -- 172.21.15.131:0/402444022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c0ce480 msgr2=0x7fbb7c06dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:15.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.478+0000 7fbb82000640 1 --2- 172.21.15.131:0/402444022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c0ce480 0x7fbb7c06dac0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0717f0 tx=0x7fbb7402a6f0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.478+0000 7fbb82000640 1 -- 172.21.15.131:0/402444022 shutdown_connections 2024-04-28T12:37:15.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.478+0000 7fbb82000640 1 --2- 172.21.15.131:0/402444022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c0ce480 0x7fbb7c06dac0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.478+0000 7fbb82000640 1 --2- 172.21.15.131:0/402444022 >> v2:172.21.15.157:3300/0 conn(0x7fbb7c0cd180 0x7fbb7c072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.478+0000 7fbb82000640 1 -- 172.21.15.131:0/402444022 >> 172.21.15.131:0/402444022 conn(0x7fbb7c068090 msgr2=0x7fbb7c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:15.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.478+0000 7fbb82000640 1 -- 172.21.15.131:0/402444022 shutdown_connections 2024-04-28T12:37:15.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.478+0000 7fbb82000640 1 -- 172.21.15.131:0/402444022 wait complete. 2024-04-28T12:37:15.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.480+0000 7fbb82000640 1 Processor -- start 2024-04-28T12:37:15.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.480+0000 7fbb82000640 1 -- start start 2024-04-28T12:37:15.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.481+0000 7fbb82000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb7c0cd180 0x7fbb7c144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:15.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.481+0000 7fbb82000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7c0ce480 0x7fbb7c1e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:15.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.481+0000 7fbb82000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c15dc50 0x7fbb7c1e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:15.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.481+0000 7fbb82000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c15e0d0 con 0x7fbb7c15dc50 2024-04-28T12:37:15.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.481+0000 7fbb82000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c073da0 con 0x7fbb7c0cd180 2024-04-28T12:37:15.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.481+0000 7fbb82000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7c06e000 con 0x7fbb7c0ce480 2024-04-28T12:37:15.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.481+0000 7fbb81600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c15dc50 0x7fbb7c1e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:15.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.481+0000 7fbb81600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c15dc50 0x7fbb7c1e82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59932/0 (socket says 172.21.15.131:59932) 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.481+0000 7fbb81600640 1 -- 172.21.15.131:0/2476413118 learned_addr learned my addr 172.21.15.131:0/2476413118 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.482+0000 7fbb80c00640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb7c0cd180 0x7fbb7c144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.482+0000 7fbb73e00640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7c0ce480 0x7fbb7c1e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.482+0000 7fbb81600640 1 -- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7c0ce480 msgr2=0x7fbb7c1e7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.482+0000 7fbb81600640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7c0ce480 0x7fbb7c1e7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.482+0000 7fbb81600640 1 -- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb7c0cd180 msgr2=0x7fbb7c144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.482+0000 7fbb81600640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb7c0cd180 0x7fbb7c144290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.482+0000 7fbb81600640 1 -- 172.21.15.131:0/2476413118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c1e88e0 con 0x7fbb7c15dc50 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.482+0000 7fbb81600640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c15dc50 0x7fbb7c1e82a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009f40 tx=0x7fbb74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.482+0000 7fbb79600640 1 -- 172.21.15.131:0/2476413118 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74041020 con 0x7fbb7c15dc50 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.482+0000 7fbb79600640 1 -- 172.21.15.131:0/2476413118 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb740077a0 con 0x7fbb7c15dc50 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.482+0000 7fbb82000640 1 -- 172.21.15.131:0/2476413118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c1e8bc0 con 0x7fbb7c15dc50 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.482+0000 7fbb79600640 1 -- 172.21.15.131:0/2476413118 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74036040 con 0x7fbb7c15dc50 2024-04-28T12:37:15.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.483+0000 7fbb82000640 1 -- 172.21.15.131:0/2476413118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c1e9180 con 0x7fbb7c15dc50 2024-04-28T12:37:15.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.484+0000 7fbb82000640 1 -- 172.21.15.131:0/2476413118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb7c1e94b0 con 0x7fbb7c15dc50 2024-04-28T12:37:15.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.484+0000 7fbb79600640 1 -- 172.21.15.131:0/2476413118 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fbb74044020 con 0x7fbb7c15dc50 2024-04-28T12:37:15.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.485+0000 7fbb79600640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbb5806c860 0x7fbb5806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:15.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.486+0000 7fbb79600640 1 -- 172.21.15.131:0/2476413118 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7fbb740ac4a0 con 0x7fbb7c15dc50 2024-04-28T12:37:15.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.486+0000 7fbb80c00640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbb5806c860 0x7fbb5806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:15.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.487+0000 7fbb80c00640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbb5806c860 0x7fbb5806ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009890 tx=0x7fbb6c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:15.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.490+0000 7fbb79600640 1 -- 172.21.15.131:0/2476413118 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb7404b030 con 0x7fbb7c15dc50 2024-04-28T12:37:15.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.713+0000 7fbb82000640 1 -- 172.21.15.131:0/2476413118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb7c069350 con 0x7fbb7c15dc50 2024-04-28T12:37:15.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.714+0000 7fbb79600640 1 -- 172.21.15.131:0/2476413118 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7fbb7407ae60 con 0x7fbb7c15dc50 2024-04-28T12:37:15.710 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:15.710 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1064054651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1064054651},{"type":"v1","addr":"172.21.15.131:6834","nonce":1064054651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-28T12:37:15.711 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 2 2024-04-28T12:37:15.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.716+0000 7fbb82000640 1 -- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbb5806c860 msgr2=0x7fbb5806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:15.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.716+0000 7fbb82000640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbb5806c860 0x7fbb5806ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009890 tx=0x7fbb6c008040 comp rx=0 tx=0).stop 2024-04-28T12:37:15.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.716+0000 7fbb82000640 1 -- 172.21.15.131:0/2476413118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c15dc50 msgr2=0x7fbb7c1e82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:15.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.716+0000 7fbb82000640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c15dc50 0x7fbb7c1e82a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009f40 tx=0x7fbb74004a40 comp rx=0 tx=0).stop 2024-04-28T12:37:15.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.716+0000 7fbb82000640 1 -- 172.21.15.131:0/2476413118 shutdown_connections 2024-04-28T12:37:15.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.716+0000 7fbb82000640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbb5806c860 0x7fbb5806ed20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.716+0000 7fbb82000640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb7c15dc50 0x7fbb7c1e82a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.716+0000 7fbb82000640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb7c0ce480 0x7fbb7c1e7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.716+0000 7fbb82000640 1 --2- 172.21.15.131:0/2476413118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb7c0cd180 0x7fbb7c144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:15.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.716+0000 7fbb82000640 1 -- 172.21.15.131:0/2476413118 >> 172.21.15.131:0/2476413118 conn(0x7fbb7c068090 msgr2=0x7fbb7c0b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:15.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.716+0000 7fbb82000640 1 -- 172.21.15.131:0/2476413118 shutdown_connections 2024-04-28T12:37:15.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:15.717+0000 7fbb82000640 1 -- 172.21.15.131:0/2476413118 wait complete. 2024-04-28T12:37:17.732 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:37:17.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.922+0000 7fd83e600640 1 Processor -- start 2024-04-28T12:37:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.922+0000 7fd83e600640 1 -- start start 2024-04-28T12:37:17.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.923+0000 7fd83e600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd83815cab0 0x7fd8380afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:17.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.923+0000 7fd83e600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd8380b0020 0x7fd8380d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:17.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.923+0000 7fd83e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8380d1080 0x7fd8380ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:17.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.923+0000 7fd83e600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd8380d14f0 con 0x7fd8380b0020 2024-04-28T12:37:17.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.923+0000 7fd83e600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd8380d0900 con 0x7fd83815cab0 2024-04-28T12:37:17.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.923+0000 7fd83e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd8381635a0 con 0x7fd8380d1080 2024-04-28T12:37:17.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.924+0000 7fd82fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd8380b0020 0x7fd8380d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:17.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.924+0000 7fd82fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd8380b0020 0x7fd8380d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59944/0 (socket says 172.21.15.131:59944) 2024-04-28T12:37:17.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.924+0000 7fd82fe00640 1 -- 172.21.15.131:0/4076630571 learned_addr learned my addr 172.21.15.131:0/4076630571 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:17.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.924+0000 7fd837400640 1 --2- 172.21.15.131:0/4076630571 >> v2:172.21.15.157:3300/0 conn(0x7fd83815cab0 0x7fd8380afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:17.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.924+0000 7fd837e00640 1 --2- 172.21.15.131:0/4076630571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8380d1080 0x7fd8380ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:17.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.924+0000 7fd82fe00640 1 -- 172.21.15.131:0/4076630571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8380d1080 msgr2=0x7fd8380ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:17.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.924+0000 7fd82fe00640 1 --2- 172.21.15.131:0/4076630571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8380d1080 0x7fd8380ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:17.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.924+0000 7fd82fe00640 1 -- 172.21.15.131:0/4076630571 >> v2:172.21.15.157:3300/0 conn(0x7fd83815cab0 msgr2=0x7fd8380afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:17.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.924+0000 7fd82fe00640 1 --2- 172.21.15.131:0/4076630571 >> v2:172.21.15.157:3300/0 conn(0x7fd83815cab0 0x7fd8380afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:17.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.924+0000 7fd82fe00640 1 -- 172.21.15.131:0/4076630571 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8380b0650 con 0x7fd8380b0020 2024-04-28T12:37:17.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.925+0000 7fd82fe00640 1 --2- 172.21.15.131:0/4076630571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380b0020 0x7fd8380d0b40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd83806d120 tx=0x7fd82002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ecd96133b6d593 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:17.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.926+0000 7fd836a00640 1 -- 172.21.15.131:0/4076630571 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820037070 con 0x7fd8380b0020 2024-04-28T12:37:17.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.926+0000 7fd836a00640 1 -- 172.21.15.131:0/4076630571 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd820032440 con 0x7fd8380b0020 2024-04-28T12:37:17.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.926+0000 7fd836a00640 1 -- 172.21.15.131:0/4076630571 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820032810 con 0x7fd8380b0020 2024-04-28T12:37:17.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.926+0000 7fd83e600640 1 -- 172.21.15.131:0/4076630571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380b0020 msgr2=0x7fd8380d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:17.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.926+0000 7fd83e600640 1 --2- 172.21.15.131:0/4076630571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380b0020 0x7fd8380d0b40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd83806d120 tx=0x7fd82002a1e0 comp rx=0 tx=0).stop 2024-04-28T12:37:17.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.926+0000 7fd83e600640 1 -- 172.21.15.131:0/4076630571 shutdown_connections 2024-04-28T12:37:17.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.926+0000 7fd83e600640 1 --2- 172.21.15.131:0/4076630571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380b0020 0x7fd8380d0b40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:17.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.926+0000 7fd83e600640 1 --2- 172.21.15.131:0/4076630571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8380d1080 0x7fd8380ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:17.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.926+0000 7fd83e600640 1 --2- 172.21.15.131:0/4076630571 >> v2:172.21.15.157:3300/0 conn(0x7fd83815cab0 0x7fd8380afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:17.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.926+0000 7fd83e600640 1 -- 172.21.15.131:0/4076630571 >> 172.21.15.131:0/4076630571 conn(0x7fd8380b2900 msgr2=0x7fd8380b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:17.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.926+0000 7fd83e600640 1 -- 172.21.15.131:0/4076630571 shutdown_connections 2024-04-28T12:37:17.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.926+0000 7fd83e600640 1 -- 172.21.15.131:0/4076630571 wait complete. 2024-04-28T12:37:17.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.928+0000 7fd83e600640 1 Processor -- start 2024-04-28T12:37:17.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.928+0000 7fd83e600640 1 -- start start 2024-04-28T12:37:17.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.929+0000 7fd83e600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380b0020 0x7fd8381f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:17.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.929+0000 7fd83e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8380d1080 0x7fd8381f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:17.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.929+0000 7fd83e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83815cab0 0x7fd8381f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:17.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.929+0000 7fd83e600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd838162f50 con 0x7fd8380b0020 2024-04-28T12:37:17.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.929+0000 7fd83e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd838162c70 con 0x7fd8380d1080 2024-04-28T12:37:17.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.929+0000 7fd83e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd838163230 con 0x7fd83815cab0 2024-04-28T12:37:17.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.929+0000 7fd837400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380b0020 0x7fd8381f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:17.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.929+0000 7fd837400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380b0020 0x7fd8381f4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59958/0 (socket says 172.21.15.131:59958) 2024-04-28T12:37:17.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.929+0000 7fd837400640 1 -- 172.21.15.131:0/3873547928 learned_addr learned my addr 172.21.15.131:0/3873547928 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:17.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.930+0000 7fd837400640 1 -- 172.21.15.131:0/3873547928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83815cab0 msgr2=0x7fd8381f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:17.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.930+0000 7fd837400640 1 --2- 172.21.15.131:0/3873547928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83815cab0 0x7fd8381f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:17.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.930+0000 7fd837400640 1 -- 172.21.15.131:0/3873547928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8380d1080 msgr2=0x7fd8381f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:17.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.930+0000 7fd837400640 1 --2- 172.21.15.131:0/3873547928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8380d1080 0x7fd8381f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:17.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.930+0000 7fd837400640 1 -- 172.21.15.131:0/3873547928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8381f67a0 con 0x7fd8380b0020 2024-04-28T12:37:17.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.930+0000 7fd837400640 1 --2- 172.21.15.131:0/3873547928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380b0020 0x7fd8381f4a60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd82800d5f0 tx=0x7fd82800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:17.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.930+0000 7fd82f400640 1 -- 172.21.15.131:0/3873547928 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8280125f0 con 0x7fd8380b0020 2024-04-28T12:37:17.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.930+0000 7fd82f400640 1 -- 172.21.15.131:0/3873547928 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd828019070 con 0x7fd8380b0020 2024-04-28T12:37:17.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.930+0000 7fd82f400640 1 -- 172.21.15.131:0/3873547928 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8280129d0 con 0x7fd8380b0020 2024-04-28T12:37:17.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.930+0000 7fd83e600640 1 -- 172.21.15.131:0/3873547928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8381f6ae0 con 0x7fd8380b0020 2024-04-28T12:37:17.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.931+0000 7fd83e600640 1 -- 172.21.15.131:0/3873547928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8381f7320 con 0x7fd8380b0020 2024-04-28T12:37:17.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.932+0000 7fd83e600640 1 -- 172.21.15.131:0/3873547928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd838069350 con 0x7fd8380b0020 2024-04-28T12:37:17.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.933+0000 7fd82f400640 1 -- 172.21.15.131:0/3873547928 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fd828012b70 con 0x7fd8380b0020 2024-04-28T12:37:17.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.934+0000 7fd82f400640 1 --2- 172.21.15.131:0/3873547928 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd81406c860 0x7fd81406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:17.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.934+0000 7fd82f400640 1 -- 172.21.15.131:0/3873547928 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd82808bbd0 con 0x7fd8380b0020 2024-04-28T12:37:17.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.934+0000 7fd82fe00640 1 --2- 172.21.15.131:0/3873547928 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd81406c860 0x7fd81406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:17.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.935+0000 7fd82fe00640 1 --2- 172.21.15.131:0/3873547928 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd81406c860 0x7fd81406ed20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd820034000 tx=0x7fd82002c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:17.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:17.939+0000 7fd82f400640 1 -- 172.21.15.131:0/3873547928 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8381f6ae0 con 0x7fd8380b0020 2024-04-28T12:37:18.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.183+0000 7fd83e600640 1 -- 172.21.15.131:0/3873547928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8381f7d80 con 0x7fd8380b0020 2024-04-28T12:37:18.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.184+0000 7fd82f400640 1 -- 172.21.15.131:0/3873547928 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3874 (secure 0 0 0) 0x7fd82805a3a0 con 0x7fd8380b0020 2024-04-28T12:37:18.181 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:18.181 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1064054651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1064054651},{"type":"v1","addr":"172.21.15.131:6834","nonce":1064054651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/398655652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":398655652},{"type":"v1","addr":"172.21.15.131:6835","nonce":398655652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2619661531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2619661531},{"type":"v1","addr":"172.21.15.157:6837","nonce":2619661531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1937780695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1937780695},{"type":"v1","addr":"172.21.15.157:6835","nonce":1937780695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-28T12:37:18.182 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 3 2024-04-28T12:37:18.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.187+0000 7fd83e600640 1 -- 172.21.15.131:0/3873547928 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd81406c860 msgr2=0x7fd81406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.187+0000 7fd83e600640 1 --2- 172.21.15.131:0/3873547928 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd81406c860 0x7fd81406ed20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd820034000 tx=0x7fd82002c000 comp rx=0 tx=0).stop 2024-04-28T12:37:18.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.187+0000 7fd83e600640 1 -- 172.21.15.131:0/3873547928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380b0020 msgr2=0x7fd8381f4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.187+0000 7fd83e600640 1 --2- 172.21.15.131:0/3873547928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380b0020 0x7fd8381f4a60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd82800d5f0 tx=0x7fd82800dab0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.187+0000 7fd83e600640 1 -- 172.21.15.131:0/3873547928 shutdown_connections 2024-04-28T12:37:18.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.187+0000 7fd83e600640 1 --2- 172.21.15.131:0/3873547928 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd81406c860 0x7fd81406ed20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.187+0000 7fd83e600640 1 --2- 172.21.15.131:0/3873547928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd83815cab0 0x7fd8381f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.187+0000 7fd83e600640 1 --2- 172.21.15.131:0/3873547928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8380d1080 0x7fd8381f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.188+0000 7fd83e600640 1 --2- 172.21.15.131:0/3873547928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8380b0020 0x7fd8381f4a60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.188+0000 7fd83e600640 1 -- 172.21.15.131:0/3873547928 >> 172.21.15.131:0/3873547928 conn(0x7fd8380b2900 msgr2=0x7fd8380b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:18.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.188+0000 7fd83e600640 1 -- 172.21.15.131:0/3873547928 shutdown_connections 2024-04-28T12:37:18.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.188+0000 7fd83e600640 1 -- 172.21.15.131:0/3873547928 wait complete. 2024-04-28T12:37:18.203 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:18.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.408+0000 7f4497e00640 1 Processor -- start 2024-04-28T12:37:18.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.409+0000 7f4497e00640 1 -- start start 2024-04-28T12:37:18.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.410+0000 7f4497e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44980d5750 0x7f44980d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:18.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.410+0000 7f4497e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44980d0480 con 0x7f44980d1620 2024-04-28T12:37:18.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.410+0000 7f4497e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44980d0720 con 0x7f44980b5b80 2024-04-28T12:37:18.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.410+0000 7f4497e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f44980d5be0 con 0x7f44980d5750 2024-04-28T12:37:18.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.410+0000 7f4496a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f44980d1620 0x7f44980d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:57720/0 (socket says 172.21.15.131:57720) 2024-04-28T12:37:18.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.410+0000 7f4496a00640 1 -- 172.21.15.131:0/82466312 learned_addr learned my addr 172.21.15.131:0/82466312 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:18.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.411+0000 7f4497400640 1 --2- 172.21.15.131:0/82466312 >> v2:172.21.15.157:3300/0 conn(0x7f44980d5750 0x7f44980d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:18.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.411+0000 7f4495600640 1 -- 172.21.15.131:0/82466312 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2969514817 0 0) 0x7f44980d0480 con 0x7f44980d1620 2024-04-28T12:37:18.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.411+0000 7f4495600640 1 -- 172.21.15.131:0/82466312 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4478003520 con 0x7f44980d1620 2024-04-28T12:37:18.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.411+0000 7f4495600640 1 -- 172.21.15.131:0/82466312 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1948781282 0 0) 0x7f44980d0720 con 0x7f44980b5b80 2024-04-28T12:37:18.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.411+0000 7f4495600640 1 -- 172.21.15.131:0/82466312 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44980d0480 con 0x7f44980b5b80 2024-04-28T12:37:18.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.411+0000 7f4495600640 1 -- 172.21.15.131:0/82466312 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3711853030 0 0) 0x7f4478003520 con 0x7f44980d1620 2024-04-28T12:37:18.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.411+0000 7f4497400640 1 -- 172.21.15.131:0/82466312 >> v1:172.21.15.157:6789/0 conn(0x7f44980b5b80 legacy=0x7f44980d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.411+0000 7f4497400640 1 -- 172.21.15.131:0/82466312 >> v1:172.21.15.131:6789/0 conn(0x7f44980d1620 legacy=0x7f44980d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.411+0000 7f4497400640 1 -- 172.21.15.131:0/82466312 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44980d0a00 con 0x7f44980d5750 2024-04-28T12:37:18.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.412+0000 7f4497400640 1 --2- 172.21.15.131:0/82466312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44980d5750 0x7f44980d2f10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4490009720 tx=0x7f449002a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a73be249fc1e001f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:18.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.413+0000 7f4495600640 1 -- 172.21.15.131:0/82466312 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490037070 con 0x7f44980d5750 2024-04-28T12:37:18.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.413+0000 7f4495600640 1 -- 172.21.15.131:0/82466312 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449002ab60 con 0x7f44980d5750 2024-04-28T12:37:18.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.413+0000 7f4495600640 1 -- 172.21.15.131:0/82466312 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488001670 con 0x7f44980d5750 2024-04-28T12:37:18.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.413+0000 7f4497e00640 1 -- 172.21.15.131:0/82466312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44980d5750 msgr2=0x7f44980d2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.413+0000 7f4497e00640 1 --2- 172.21.15.131:0/82466312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44980d5750 0x7f44980d2f10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4490009720 tx=0x7f449002a0d0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.414+0000 7f4497e00640 1 -- 172.21.15.131:0/82466312 shutdown_connections 2024-04-28T12:37:18.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.414+0000 7f4497e00640 1 --2- 172.21.15.131:0/82466312 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44980d5750 0x7f44980d2f10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.414+0000 7f4497e00640 1 -- 172.21.15.131:0/82466312 >> 172.21.15.131:0/82466312 conn(0x7f44980d4c30 msgr2=0x7f44980d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:18.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.414+0000 7f4497e00640 1 -- 172.21.15.131:0/82466312 shutdown_connections 2024-04-28T12:37:18.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.414+0000 7f4497e00640 1 -- 172.21.15.131:0/82466312 wait complete. 2024-04-28T12:37:18.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.415+0000 7f4497e00640 1 Processor -- start 2024-04-28T12:37:18.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.415+0000 7f4497e00640 1 -- start start 2024-04-28T12:37:18.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.416+0000 7f4497e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44980b5b80 0x7f44981f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:18.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.416+0000 7f4497e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44980d1620 0x7f44980b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:18.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.416+0000 7f4497e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44980d5750 0x7f4498159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:18.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.416+0000 7f4497e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f44980d3450 con 0x7f44980d1620 2024-04-28T12:37:18.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.416+0000 7f4497e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44980d0ce0 con 0x7f44980d5750 2024-04-28T12:37:18.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.416+0000 7f4497e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44980d35d0 con 0x7f44980b5b80 2024-04-28T12:37:18.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.417+0000 7f4496000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44980d1620 0x7f44980b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:18.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.417+0000 7f4496000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44980d1620 0x7f44980b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59972/0 (socket says 172.21.15.131:59972) 2024-04-28T12:37:18.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.417+0000 7f4496000640 1 -- 172.21.15.131:0/1488083588 learned_addr learned my addr 172.21.15.131:0/1488083588 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:18.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.417+0000 7f4497400640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44980d5750 0x7f4498159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:18.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.417+0000 7f4496a00640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44980b5b80 0x7f44981f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.417+0000 7f4496000640 1 -- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44980b5b80 msgr2=0x7f44981f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.417+0000 7f4496000640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44980b5b80 0x7f44981f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.417+0000 7f4496000640 1 -- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44980d5750 msgr2=0x7f4498159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.417+0000 7f4496000640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44980d5750 0x7f4498159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.417+0000 7f4496000640 1 -- 172.21.15.131:0/1488083588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44980ba6d0 con 0x7f44980d1620 2024-04-28T12:37:18.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.417+0000 7f4496000640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44980d1620 0x7f44980b84a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f448000cb60 tx=0x7f448000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:18.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.418+0000 7f448f400640 1 -- 172.21.15.131:0/1488083588 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480018070 con 0x7f44980d1620 2024-04-28T12:37:18.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.418+0000 7f448f400640 1 -- 172.21.15.131:0/1488083588 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4480006d30 con 0x7f44980d1620 2024-04-28T12:37:18.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.418+0000 7f448f400640 1 -- 172.21.15.131:0/1488083588 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448000ad20 con 0x7f44980d1620 2024-04-28T12:37:18.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.418+0000 7f4497e00640 1 -- 172.21.15.131:0/1488083588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44980b79c0 con 0x7f44980d1620 2024-04-28T12:37:18.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.418+0000 7f4497e00640 1 -- 172.21.15.131:0/1488083588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44980b7f80 con 0x7f44980d1620 2024-04-28T12:37:18.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.419+0000 7f4497e00640 1 -- 172.21.15.131:0/1488083588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44980b8180 con 0x7f44980d1620 2024-04-28T12:37:18.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.420+0000 7f448f400640 1 -- 172.21.15.131:0/1488083588 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f448001b070 con 0x7f44980d1620 2024-04-28T12:37:18.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.421+0000 7f448f400640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f447406c860 0x7f447406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:18.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.421+0000 7f448f400640 1 -- 172.21.15.131:0/1488083588 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f448008b070 con 0x7f44980d1620 2024-04-28T12:37:18.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.421+0000 7f4496a00640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f447406c860 0x7f447406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:18.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.422+0000 7f4496a00640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f447406c860 0x7f447406ed20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4488000f60 tx=0x7f4488000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:18.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.425+0000 7f448f400640 1 -- 172.21.15.131:0/1488083588 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f44980b8180 con 0x7f44980d1620 2024-04-28T12:37:18.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.632+0000 7f4497e00640 1 -- 172.21.15.131:0/1488083588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4498069350 con 0x7f44980d1620 2024-04-28T12:37:18.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.635+0000 7f448f400640 1 -- 172.21.15.131:0/1488083588 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12908 (secure 0 0 0) 0x7f4480059270 con 0x7f44980d1620 2024-04-28T12:37:18.632 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:18.632 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":16,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:17.682121+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:18.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.637+0000 7f4497e00640 1 -- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f447406c860 msgr2=0x7f447406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.637+0000 7f4497e00640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f447406c860 0x7f447406ed20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4488000f60 tx=0x7f4488000ed0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.637+0000 7f4497e00640 1 -- 172.21.15.131:0/1488083588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44980d1620 msgr2=0x7f44980b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.637+0000 7f4497e00640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44980d1620 0x7f44980b84a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f448000cb60 tx=0x7f448000a430 comp rx=0 tx=0).stop 2024-04-28T12:37:18.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.638+0000 7f4497e00640 1 -- 172.21.15.131:0/1488083588 shutdown_connections 2024-04-28T12:37:18.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.638+0000 7f4497e00640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f447406c860 0x7f447406ed20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.638+0000 7f4497e00640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44980d5750 0x7f4498159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.638+0000 7f4497e00640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44980d1620 0x7f44980b84a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.638+0000 7f4497e00640 1 --2- 172.21.15.131:0/1488083588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44980b5b80 0x7f44981f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.638+0000 7f4497e00640 1 -- 172.21.15.131:0/1488083588 >> 172.21.15.131:0/1488083588 conn(0x7f44980d4c30 msgr2=0x7f44980d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:18.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.638+0000 7f4497e00640 1 -- 172.21.15.131:0/1488083588 shutdown_connections 2024-04-28T12:37:18.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.638+0000 7f4497e00640 1 -- 172.21.15.131:0/1488083588 wait complete. 2024-04-28T12:37:18.653 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T12:37:18.653 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:37:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.864+0000 7fe689600640 1 Processor -- start 2024-04-28T12:37:18.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.864+0000 7fe689600640 1 -- start start 2024-04-28T12:37:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.866+0000 7fe689600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6840b54b0 0x7fe6841604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.866+0000 7fe689600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe68406df40 0x7fe68415d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.866+0000 7fe689600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6840b3d80 con 0x7fe6841458e0 2024-04-28T12:37:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.866+0000 7fe689600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe6840b4060 con 0x7fe68406df40 2024-04-28T12:37:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.866+0000 7fe689600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe6840b4340 con 0x7fe6840b54b0 2024-04-28T12:37:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.866+0000 7fe683400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe68406df40 0x7fe68415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.866+0000 7fe683400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe68406df40 0x7fe68415d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59984/0 (socket says 172.21.15.131:59984) 2024-04-28T12:37:18.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.866+0000 7fe683400640 1 -- 172.21.15.131:0/1626415303 learned_addr learned my addr 172.21.15.131:0/1626415303 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.866+0000 7fe682000640 1 --2- 172.21.15.131:0/1626415303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6840b54b0 0x7fe6841604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.866+0000 7fe683400640 1 -- 172.21.15.131:0/1626415303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6840b54b0 msgr2=0x7fe6841604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.866+0000 7fe683400640 1 --2- 172.21.15.131:0/1626415303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6840b54b0 0x7fe6841604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.867+0000 7fe683400640 1 -- 172.21.15.131:0/1626415303 >> v1:172.21.15.157:6789/0 conn(0x7fe6841458e0 legacy=0x7fe6840b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.867+0000 7fe683400640 1 -- 172.21.15.131:0/1626415303 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6840b4620 con 0x7fe68406df40 2024-04-28T12:37:18.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.868+0000 7fe683400640 1 --2- 172.21.15.131:0/1626415303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe68406df40 0x7fe68415d700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009e50 tx=0x7fe67c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d8c91be996d28d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:18.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.868+0000 7fe681600640 1 -- 172.21.15.131:0/1626415303 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c036070 con 0x7fe68406df40 2024-04-28T12:37:18.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.868+0000 7fe681600640 1 -- 172.21.15.131:0/1626415303 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe67c031600 con 0x7fe68406df40 2024-04-28T12:37:18.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.868+0000 7fe681600640 1 -- 172.21.15.131:0/1626415303 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c0319d0 con 0x7fe68406df40 2024-04-28T12:37:18.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.868+0000 7fe689600640 1 -- 172.21.15.131:0/1626415303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe68406df40 msgr2=0x7fe68415d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.868+0000 7fe689600640 1 --2- 172.21.15.131:0/1626415303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe68406df40 0x7fe68415d700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009e50 tx=0x7fe67c02a6a0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.869+0000 7fe689600640 1 -- 172.21.15.131:0/1626415303 shutdown_connections 2024-04-28T12:37:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.869+0000 7fe689600640 1 --2- 172.21.15.131:0/1626415303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe68406df40 0x7fe68415d700 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.869+0000 7fe689600640 1 --2- 172.21.15.131:0/1626415303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6840b54b0 0x7fe6841604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.869+0000 7fe689600640 1 -- 172.21.15.131:0/1626415303 >> 172.21.15.131:0/1626415303 conn(0x7fe6840b2900 msgr2=0x7fe6840b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.869+0000 7fe689600640 1 -- 172.21.15.131:0/1626415303 shutdown_connections 2024-04-28T12:37:18.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.869+0000 7fe689600640 1 -- 172.21.15.131:0/1626415303 wait complete. 2024-04-28T12:37:18.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.870+0000 7fe689600640 1 Processor -- start 2024-04-28T12:37:18.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.871+0000 7fe689600640 1 -- start start 2024-04-28T12:37:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe689600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe68406df40 0x7fe68413f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe689600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6840b54b0 0x7fe68415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe689600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6841458e0 0x7fe68413e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe689600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe68415ddc0 con 0x7fe6840b54b0 2024-04-28T12:37:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe689600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe68415dc40 con 0x7fe68406df40 2024-04-28T12:37:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe689600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe68415df40 con 0x7fe6841458e0 2024-04-28T12:37:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe682000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6840b54b0 0x7fe68415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe682000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6840b54b0 0x7fe68415b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59998/0 (socket says 172.21.15.131:59998) 2024-04-28T12:37:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe682000640 1 -- 172.21.15.131:0/3152018699 learned_addr learned my addr 172.21.15.131:0/3152018699 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe683400640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6841458e0 0x7fe68413e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe682a00640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe68406df40 0x7fe68413f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:18.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe682000640 1 -- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6841458e0 msgr2=0x7fe68413e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe682000640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6841458e0 0x7fe68413e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe682000640 1 -- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe68406df40 msgr2=0x7fe68413f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe682000640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe68406df40 0x7fe68413f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe682000640 1 -- 172.21.15.131:0/3152018699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe684141770 con 0x7fe6840b54b0 2024-04-28T12:37:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe682a00640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe68406df40 0x7fe68413f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.872+0000 7fe683400640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6841458e0 0x7fe68413e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.873+0000 7fe682000640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6840b54b0 0x7fe68415b5d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe67000d800 tx=0x7fe67000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.873+0000 7fe67aa00640 1 -- 172.21.15.131:0/3152018699 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67000b6c0 con 0x7fe6840b54b0 2024-04-28T12:37:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.873+0000 7fe67aa00640 1 -- 172.21.15.131:0/3152018699 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe670019070 con 0x7fe6840b54b0 2024-04-28T12:37:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.873+0000 7fe67aa00640 1 -- 172.21.15.131:0/3152018699 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67000bac0 con 0x7fe6840b54b0 2024-04-28T12:37:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.873+0000 7fe689600640 1 -- 172.21.15.131:0/3152018699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe684141a50 con 0x7fe6840b54b0 2024-04-28T12:37:18.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.873+0000 7fe689600640 1 -- 172.21.15.131:0/3152018699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe684142010 con 0x7fe6840b54b0 2024-04-28T12:37:18.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.874+0000 7fe689600640 1 -- 172.21.15.131:0/3152018699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe684069350 con 0x7fe6840b54b0 2024-04-28T12:37:18.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.875+0000 7fe67aa00640 1 -- 172.21.15.131:0/3152018699 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fe67000bc60 con 0x7fe6840b54b0 2024-04-28T12:37:18.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.876+0000 7fe67aa00640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe66406c860 0x7fe66406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:18.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.876+0000 7fe67aa00640 1 -- 172.21.15.131:0/3152018699 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe67008bc30 con 0x7fe6840b54b0 2024-04-28T12:37:18.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.877+0000 7fe682a00640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe66406c860 0x7fe66406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:18.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.878+0000 7fe682a00640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe66406c860 0x7fe66406ed20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe6840d03b0 tx=0x7fe674009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:18.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:18.881+0000 7fe67aa00640 1 -- 172.21.15.131:0/3152018699 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe67005a480 con 0x7fe6840b54b0 2024-04-28T12:37:19.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.084+0000 7fe689600640 1 -- 172.21.15.131:0/3152018699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe68406fac0 con 0x7fe6840b54b0 2024-04-28T12:37:19.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.085+0000 7fe67aa00640 1 -- 172.21.15.131:0/3152018699 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fe670059d80 con 0x7fe6840b54b0 2024-04-28T12:37:19.082 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:37:19.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.087+0000 7fe689600640 1 -- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe66406c860 msgr2=0x7fe66406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:19.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.087+0000 7fe689600640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe66406c860 0x7fe66406ed20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe6840d03b0 tx=0x7fe674009040 comp rx=0 tx=0).stop 2024-04-28T12:37:19.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.087+0000 7fe689600640 1 -- 172.21.15.131:0/3152018699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6840b54b0 msgr2=0x7fe68415b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:19.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.087+0000 7fe689600640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6840b54b0 0x7fe68415b5d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe67000d800 tx=0x7fe67000dcc0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.087+0000 7fe689600640 1 -- 172.21.15.131:0/3152018699 shutdown_connections 2024-04-28T12:37:19.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.087+0000 7fe689600640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe66406c860 0x7fe66406ed20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.087+0000 7fe689600640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6841458e0 0x7fe68413e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.087+0000 7fe689600640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6840b54b0 0x7fe68415b5d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.087+0000 7fe689600640 1 --2- 172.21.15.131:0/3152018699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe68406df40 0x7fe68413f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.087+0000 7fe689600640 1 -- 172.21.15.131:0/3152018699 >> 172.21.15.131:0/3152018699 conn(0x7fe6840b2900 msgr2=0x7fe6840b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:19.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.088+0000 7fe689600640 1 -- 172.21.15.131:0/3152018699 shutdown_connections 2024-04-28T12:37:19.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.088+0000 7fe689600640 1 -- 172.21.15.131:0/3152018699 wait complete. 2024-04-28T12:37:19.104 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-28T12:37:19.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.307+0000 7fda56000640 1 Processor -- start 2024-04-28T12:37:19.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.307+0000 7fda56000640 1 -- start start 2024-04-28T12:37:19.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.309+0000 7fda56000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fda50070ca0 0x7fda500af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:19.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.309+0000 7fda56000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fda501606d0 0x7fda50160b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.309+0000 7fda56000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda5015fd60 con 0x7fda5015f510 2024-04-28T12:37:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.309+0000 7fda56000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fda50071080 con 0x7fda501606d0 2024-04-28T12:37:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.309+0000 7fda56000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fda50160040 con 0x7fda50070ca0 2024-04-28T12:37:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.309+0000 7fda4fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fda501606d0 0x7fda50160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.309+0000 7fda4fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fda501606d0 0x7fda50160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60004/0 (socket says 172.21.15.131:60004) 2024-04-28T12:37:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.309+0000 7fda4fe00640 1 -- 172.21.15.131:0/4184750716 learned_addr learned my addr 172.21.15.131:0/4184750716 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.309+0000 7fda46a00640 1 --1- 172.21.15.131:0/4184750716 >> v1:172.21.15.131:6789/0 conn(0x7fda5015f510 0x7fda500b54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:57734/0 (socket says 172.21.15.131:57734) 2024-04-28T12:37:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.309+0000 7fda4f400640 1 --2- 172.21.15.131:0/4184750716 >> v2:172.21.15.157:3300/0 conn(0x7fda50070ca0 0x7fda500af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:19.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.310+0000 7fda4fe00640 1 -- 172.21.15.131:0/4184750716 >> v2:172.21.15.157:3300/0 conn(0x7fda50070ca0 msgr2=0x7fda500af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:19.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.310+0000 7fda4fe00640 1 --2- 172.21.15.131:0/4184750716 >> v2:172.21.15.157:3300/0 conn(0x7fda50070ca0 0x7fda500af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.310+0000 7fda4fe00640 1 -- 172.21.15.131:0/4184750716 >> v1:172.21.15.131:6789/0 conn(0x7fda5015f510 legacy=0x7fda500b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:19.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.310+0000 7fda4ea00640 1 -- 172.21.15.131:0/4184750716 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996620800 0 0) 0x7fda5015fd60 con 0x7fda5015f510 2024-04-28T12:37:19.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.310+0000 7fda4fe00640 1 -- 172.21.15.131:0/4184750716 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda50160320 con 0x7fda501606d0 2024-04-28T12:37:19.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.311+0000 7fda4fe00640 1 --2- 172.21.15.131:0/4184750716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda501606d0 0x7fda50160b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fda50076290 tx=0x7fda4802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed067300d0fdf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:19.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.311+0000 7fda4ea00640 1 -- 172.21.15.131:0/4184750716 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48036070 con 0x7fda501606d0 2024-04-28T12:37:19.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.311+0000 7fda4ea00640 1 -- 172.21.15.131:0/4184750716 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda48031600 con 0x7fda501606d0 2024-04-28T12:37:19.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.311+0000 7fda4ea00640 1 -- 172.21.15.131:0/4184750716 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda480319d0 con 0x7fda501606d0 2024-04-28T12:37:19.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.312+0000 7fda56000640 1 -- 172.21.15.131:0/4184750716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda501606d0 msgr2=0x7fda50160b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:19.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.312+0000 7fda56000640 1 --2- 172.21.15.131:0/4184750716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda501606d0 0x7fda50160b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fda50076290 tx=0x7fda4802a730 comp rx=0 tx=0).stop 2024-04-28T12:37:19.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.312+0000 7fda56000640 1 -- 172.21.15.131:0/4184750716 shutdown_connections 2024-04-28T12:37:19.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.312+0000 7fda56000640 1 --2- 172.21.15.131:0/4184750716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda501606d0 0x7fda50160b10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.312+0000 7fda56000640 1 --2- 172.21.15.131:0/4184750716 >> v2:172.21.15.157:3300/0 conn(0x7fda50070ca0 0x7fda500af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.312+0000 7fda56000640 1 -- 172.21.15.131:0/4184750716 >> 172.21.15.131:0/4184750716 conn(0x7fda5006c530 msgr2=0x7fda5006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:19.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.312+0000 7fda56000640 1 -- 172.21.15.131:0/4184750716 shutdown_connections 2024-04-28T12:37:19.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.312+0000 7fda56000640 1 -- 172.21.15.131:0/4184750716 wait complete. 2024-04-28T12:37:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.314+0000 7fda56000640 1 Processor -- start 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.314+0000 7fda56000640 1 -- start start 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda56000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50070ca0 0x7fda5015c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda56000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda5015f510 0x7fda500cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda56000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda501606d0 0x7fda500ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda56000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda500b5d40 con 0x7fda50070ca0 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda56000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda500b5bc0 con 0x7fda501606d0 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda56000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fda5015f970 con 0x7fda5015f510 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda4f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50070ca0 0x7fda5015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda4f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50070ca0 0x7fda5015c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60016/0 (socket says 172.21.15.131:60016) 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda4f400640 1 -- 172.21.15.131:0/4250709450 learned_addr learned my addr 172.21.15.131:0/4250709450 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda4f400640 1 -- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda5015f510 msgr2=0x7fda500cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda4fe00640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda501606d0 0x7fda500ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda46a00640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda5015f510 0x7fda500cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda4f400640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda5015f510 0x7fda500cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda4f400640 1 -- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda501606d0 msgr2=0x7fda500ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda4f400640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda501606d0 0x7fda500ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.315+0000 7fda4f400640 1 -- 172.21.15.131:0/4250709450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda500d0800 con 0x7fda50070ca0 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.316+0000 7fda4f400640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50070ca0 0x7fda5015c000 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e9e0 tx=0x7fda4000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.316+0000 7fda47e00640 1 -- 172.21.15.131:0/4250709450 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40017070 con 0x7fda50070ca0 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.316+0000 7fda47e00640 1 -- 172.21.15.131:0/4250709450 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4000c9d0 con 0x7fda50070ca0 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.316+0000 7fda56000640 1 -- 172.21.15.131:0/4250709450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda501570f0 con 0x7fda50070ca0 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.316+0000 7fda47e00640 1 -- 172.21.15.131:0/4250709450 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4000cdc0 con 0x7fda50070ca0 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.316+0000 7fda56000640 1 -- 172.21.15.131:0/4250709450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda501576b0 con 0x7fda50070ca0 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.317+0000 7fda56000640 1 -- 172.21.15.131:0/4250709450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda50069350 con 0x7fda50070ca0 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.318+0000 7fda47e00640 1 -- 172.21.15.131:0/4250709450 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fda4001a070 con 0x7fda50070ca0 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.319+0000 7fda47e00640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fda2c06c860 0x7fda2c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.319+0000 7fda47e00640 1 -- 172.21.15.131:0/4250709450 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fda4008b720 con 0x7fda50070ca0 2024-04-28T12:37:19.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.320+0000 7fda46a00640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fda2c06c860 0x7fda2c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:19.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.321+0000 7fda46a00640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fda2c06c860 0x7fda2c06ed20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda38007fd0 tx=0x7fda38009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:19.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.324+0000 7fda47e00640 1 -- 172.21.15.131:0/4250709450 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda40059f80 con 0x7fda50070ca0 2024-04-28T12:37:19.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.528+0000 7fda56000640 1 -- 172.21.15.131:0/4250709450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fda500cd8b0 con 0x7fda50070ca0 2024-04-28T12:37:19.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.529+0000 7fda47e00640 1 -- 172.21.15.131:0/4250709450 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fda40059880 con 0x7fda50070ca0 2024-04-28T12:37:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:19.526 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-04-28T12:37:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:37:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-04-28T12:37:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-04-28T12:37:19.526 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:37:19.526 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-04-28T12:37:19.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.531+0000 7fda56000640 1 -- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fda2c06c860 msgr2=0x7fda2c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:19.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.531+0000 7fda56000640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fda2c06c860 0x7fda2c06ed20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda38007fd0 tx=0x7fda38009040 comp rx=0 tx=0).stop 2024-04-28T12:37:19.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.531+0000 7fda56000640 1 -- 172.21.15.131:0/4250709450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50070ca0 msgr2=0x7fda5015c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:19.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.531+0000 7fda56000640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50070ca0 0x7fda5015c000 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e9e0 tx=0x7fda4000eea0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.532+0000 7fda56000640 1 -- 172.21.15.131:0/4250709450 shutdown_connections 2024-04-28T12:37:19.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.532+0000 7fda56000640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fda2c06c860 0x7fda2c06ed20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.532+0000 7fda56000640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda501606d0 0x7fda500ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.532+0000 7fda56000640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda5015f510 0x7fda500cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.532+0000 7fda56000640 1 --2- 172.21.15.131:0/4250709450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50070ca0 0x7fda5015c000 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.532+0000 7fda56000640 1 -- 172.21.15.131:0/4250709450 >> 172.21.15.131:0/4250709450 conn(0x7fda5006c530 msgr2=0x7fda500b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:19.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.532+0000 7fda56000640 1 -- 172.21.15.131:0/4250709450 shutdown_connections 2024-04-28T12:37:19.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.532+0000 7fda56000640 1 -- 172.21.15.131:0/4250709450 wait complete. 2024-04-28T12:37:19.547 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-28T12:37:19.547 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-28T12:37:19.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.748+0000 7f9189800640 1 Processor -- start 2024-04-28T12:37:19.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.749+0000 7f9189800640 1 -- start start 2024-04-28T12:37:19.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.750+0000 7f9189800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9184145780 0x7f918406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:19.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.750+0000 7f9189800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91840733d0 0x7f91841657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:19.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.750+0000 7f9189800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f918406f7e0 con 0x7f918406f260 2024-04-28T12:37:19.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.750+0000 7f9189800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f918406fac0 con 0x7f9184145780 2024-04-28T12:37:19.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.750+0000 7f9189800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f918406fda0 con 0x7f91840733d0 2024-04-28T12:37:19.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.751+0000 7f9183e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9184145780 0x7f918406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:19.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.751+0000 7f9188e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91840733d0 0x7f91841657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:19.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.751+0000 7f9183e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9184145780 0x7f918406ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:60780/0 (socket says 172.21.15.131:60780) 2024-04-28T12:37:19.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.751+0000 7f9183e00640 1 -- 172.21.15.131:0/2051335768 learned_addr learned my addr 172.21.15.131:0/2051335768 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:19.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.751+0000 7f9183e00640 1 -- 172.21.15.131:0/2051335768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91840733d0 msgr2=0x7f91841657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:19.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.752+0000 7f9183e00640 1 --2- 172.21.15.131:0/2051335768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91840733d0 0x7f91841657a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.752+0000 7f9183e00640 1 -- 172.21.15.131:0/2051335768 >> v1:172.21.15.157:6789/0 conn(0x7f918406f260 legacy=0x7f9184072cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:19.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.752+0000 7f9183e00640 1 -- 172.21.15.131:0/2051335768 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9184070080 con 0x7f9184145780 2024-04-28T12:37:19.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.753+0000 7f9183e00640 1 --2- 172.21.15.131:0/2051335768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184145780 0x7f918406ed20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9174009870 tx=0x7f917402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=235577b6f2f7e771 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:19.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.753+0000 7f9182a00640 1 -- 172.21.15.131:0/2051335768 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174037070 con 0x7f9184145780 2024-04-28T12:37:19.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.753+0000 7f9182a00640 1 -- 172.21.15.131:0/2051335768 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9174032440 con 0x7f9184145780 2024-04-28T12:37:19.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.754+0000 7f9182a00640 1 -- 172.21.15.131:0/2051335768 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174032810 con 0x7f9184145780 2024-04-28T12:37:19.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.754+0000 7f9189800640 1 -- 172.21.15.131:0/2051335768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184145780 msgr2=0x7f918406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:19.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.754+0000 7f9189800640 1 --2- 172.21.15.131:0/2051335768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184145780 0x7f918406ed20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9174009870 tx=0x7f917402a1c0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.754+0000 7f9189800640 1 -- 172.21.15.131:0/2051335768 shutdown_connections 2024-04-28T12:37:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.754+0000 7f9189800640 1 --2- 172.21.15.131:0/2051335768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91840733d0 0x7f91841657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.754+0000 7f9189800640 1 --2- 172.21.15.131:0/2051335768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9184145780 0x7f918406ed20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.754+0000 7f9189800640 1 -- 172.21.15.131:0/2051335768 >> 172.21.15.131:0/2051335768 conn(0x7f91840b3730 msgr2=0x7f91840b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.754+0000 7f9189800640 1 -- 172.21.15.131:0/2051335768 shutdown_connections 2024-04-28T12:37:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.754+0000 7f9189800640 1 -- 172.21.15.131:0/2051335768 wait complete. 2024-04-28T12:37:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.756+0000 7f9189800640 1 Processor -- start 2024-04-28T12:37:19.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.756+0000 7f9189800640 1 -- start start 2024-04-28T12:37:19.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.757+0000 7f9189800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f918406f260 0x7f91841f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:19.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.757+0000 7f9189800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91840733d0 0x7f91840abf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:19.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.757+0000 7f9189800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184145780 0x7f918406c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:19.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.757+0000 7f9189800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9184070880 con 0x7f9184145780 2024-04-28T12:37:19.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.757+0000 7f9189800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91840b5690 con 0x7f91840733d0 2024-04-28T12:37:19.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.757+0000 7f9189800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9184070a00 con 0x7f918406f260 2024-04-28T12:37:19.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.758+0000 7f9188e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184145780 0x7f918406c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.758+0000 7f9188e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184145780 0x7f918406c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60024/0 (socket says 172.21.15.131:60024) 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.758+0000 7f9188e00640 1 -- 172.21.15.131:0/3272397364 learned_addr learned my addr 172.21.15.131:0/3272397364 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.758+0000 7f9183e00640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f918406f260 0x7f91841f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.758+0000 7f9183400640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91840733d0 0x7f91840abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.758+0000 7f9188e00640 1 -- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f918406f260 msgr2=0x7f91841f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.758+0000 7f9188e00640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f918406f260 0x7f91841f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.758+0000 7f9188e00640 1 -- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91840733d0 msgr2=0x7f91840abf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.758+0000 7f9188e00640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91840733d0 0x7f91840abf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.758+0000 7f9188e00640 1 -- 172.21.15.131:0/3272397364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91841f4fa0 con 0x7f9184145780 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.758+0000 7f9188e00640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184145780 0x7f918406c350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f917c00ef90 tx=0x7f917c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.758+0000 7f917be00640 1 -- 172.21.15.131:0/3272397364 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c018020 con 0x7f9184145780 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.759+0000 7f917be00640 1 -- 172.21.15.131:0/3272397364 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f917c014480 con 0x7f9184145780 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.759+0000 7f9189800640 1 -- 172.21.15.131:0/3272397364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91841f5280 con 0x7f9184145780 2024-04-28T12:37:19.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.759+0000 7f917be00640 1 -- 172.21.15.131:0/3272397364 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c014870 con 0x7f9184145780 2024-04-28T12:37:19.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.759+0000 7f9189800640 1 -- 172.21.15.131:0/3272397364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91841f5840 con 0x7f9184145780 2024-04-28T12:37:19.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.760+0000 7f9189800640 1 -- 172.21.15.131:0/3272397364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9184069a60 con 0x7f9184145780 2024-04-28T12:37:19.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.761+0000 7f917be00640 1 -- 172.21.15.131:0/3272397364 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f917c005d10 con 0x7f9184145780 2024-04-28T12:37:19.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.762+0000 7f917be00640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f916006c8f0 0x7f916006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:19.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.762+0000 7f917be00640 1 -- 172.21.15.131:0/3272397364 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f917c08c610 con 0x7f9184145780 2024-04-28T12:37:19.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.762+0000 7f9183e00640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f916006c8f0 0x7f916006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:19.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.763+0000 7f9183e00640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f916006c8f0 0x7f916006edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9174034000 tx=0x7f917402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:19.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.766+0000 7f917be00640 1 -- 172.21.15.131:0/3272397364 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f917c022030 con 0x7f9184145780 2024-04-28T12:37:19.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:19.968+0000 7f9189800640 1 -- 172.21.15.131:0/3272397364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f91840b44a0 con 0x7f9184145780 2024-04-28T12:37:20.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.747+0000 7f917be00640 1 -- 172.21.15.131:0/3272397364 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f917c010040 con 0x7f9184145780 2024-04-28T12:37:20.744 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-04-28T12:37:20.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.751+0000 7f9189800640 1 -- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f916006c8f0 msgr2=0x7f916006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:20.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.751+0000 7f9189800640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f916006c8f0 0x7f916006edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9174034000 tx=0x7f917402e000 comp rx=0 tx=0).stop 2024-04-28T12:37:20.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.751+0000 7f9189800640 1 -- 172.21.15.131:0/3272397364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184145780 msgr2=0x7f918406c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:20.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.751+0000 7f9189800640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184145780 0x7f918406c350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f917c00ef90 tx=0x7f917c00c490 comp rx=0 tx=0).stop 2024-04-28T12:37:20.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.751+0000 7f9189800640 1 -- 172.21.15.131:0/3272397364 shutdown_connections 2024-04-28T12:37:20.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.751+0000 7f9189800640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f916006c8f0 0x7f916006edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:20.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.751+0000 7f9189800640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9184145780 0x7f918406c350 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:20.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.751+0000 7f9189800640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91840733d0 0x7f91840abf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:20.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.751+0000 7f9189800640 1 --2- 172.21.15.131:0/3272397364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f918406f260 0x7f91841f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:20.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.751+0000 7f9189800640 1 -- 172.21.15.131:0/3272397364 >> 172.21.15.131:0/3272397364 conn(0x7f91840b3730 msgr2=0x7f918415c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:20.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.751+0000 7f9189800640 1 -- 172.21.15.131:0/3272397364 shutdown_connections 2024-04-28T12:37:20.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.751+0000 7f9189800640 1 -- 172.21.15.131:0/3272397364 wait complete. 2024-04-28T12:37:20.776 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-28T12:37:20.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.971+0000 7f9db5400640 1 Processor -- start 2024-04-28T12:37:20.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.972+0000 7f9db5400640 1 -- start start 2024-04-28T12:37:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.973+0000 7f9db5400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9db00b5250 0x7f9db00b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.973+0000 7f9db5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9db00cf330 0x7f9db00cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.973+0000 7f9db5400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db006f840 con 0x7f9db0146a30 2024-04-28T12:37:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.973+0000 7f9db5400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9db006ea80 con 0x7f9db00b5250 2024-04-28T12:37:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.973+0000 7f9db5400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9db006ed60 con 0x7f9db00cf330 2024-04-28T12:37:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.974+0000 7f9dae800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9db0146a30 0x7f9db00b4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54878/0 (socket says 172.21.15.131:54878) 2024-04-28T12:37:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.974+0000 7f9dae800640 1 -- 172.21.15.131:0/254284079 learned_addr learned my addr 172.21.15.131:0/254284079 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.974+0000 7f9da5e00640 1 --2- 172.21.15.131:0/254284079 >> v2:172.21.15.157:3300/0 conn(0x7f9db00b5250 0x7f9db00b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.974+0000 7f9daf200640 1 --2- 172.21.15.131:0/254284079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9db00cf330 0x7f9db00cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.974+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1327476949 0 0) 0x7f9db006f840 con 0x7f9db0146a30 2024-04-28T12:37:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.974+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d94003520 con 0x7f9db0146a30 2024-04-28T12:37:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.974+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2301988117 0 0) 0x7f9d94003520 con 0x7f9db0146a30 2024-04-28T12:37:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.974+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9db00cf330 msgr2=0x7f9db00cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.974+0000 7f9dade00640 1 --2- 172.21.15.131:0/254284079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9db00cf330 0x7f9db00cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.974+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 >> v2:172.21.15.157:3300/0 conn(0x7f9db00b5250 msgr2=0x7f9db00b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.974+0000 7f9dade00640 1 --2- 172.21.15.131:0/254284079 >> v2:172.21.15.157:3300/0 conn(0x7f9db00b5250 0x7f9db00b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.974+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db006f040 con 0x7f9db0146a30 2024-04-28T12:37:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f9da0001670 con 0x7f9db0146a30 2024-04-28T12:37:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 >> v1:172.21.15.131:6789/0 conn(0x7f9db0146a30 legacy=0x7f9db00b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9dade00640 1 --2- 172.21.15.131:0/254284079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d940039b0 0x7f9d94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9dade00640 1 --2- 172.21.15.131:0/254284079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d94004800 0x7f9d94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9dade00640 1 --2- 172.21.15.131:0/254284079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9db00cf330 0x7f9d940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9db006ea80 con 0x7f9d94004800 2024-04-28T12:37:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9db006ed60 con 0x7f9d940039b0 2024-04-28T12:37:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9400ba70 con 0x7f9db00cf330 2024-04-28T12:37:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9da5e00640 1 --2- 172.21.15.131:0/254284079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d94004800 0x7f9d94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9dae800640 1 --2- 172.21.15.131:0/254284079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d940039b0 0x7f9d94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9da5e00640 1 -- 172.21.15.131:0/254284079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9db00cf330 msgr2=0x7f9d940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9da5e00640 1 --2- 172.21.15.131:0/254284079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9db00cf330 0x7f9d940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9da5e00640 1 -- 172.21.15.131:0/254284079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d940039b0 msgr2=0x7f9d94003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9da5e00640 1 --2- 172.21.15.131:0/254284079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d940039b0 0x7f9d94003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9da5e00640 1 -- 172.21.15.131:0/254284079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d94003520 con 0x7f9d94004800 2024-04-28T12:37:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.975+0000 7f9dae800640 1 --2- 172.21.15.131:0/254284079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d940039b0 0x7f9d94003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.976+0000 7f9da5e00640 1 --2- 172.21.15.131:0/254284079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d94004800 0x7f9d94004c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009ac0 tx=0x7f9d94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2126a0f8d98a56a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.977+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98036070 con 0x7f9d94004800 2024-04-28T12:37:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.977+0000 7f9dade00640 1 -- 172.21.15.131:0/254284079 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d98031400 con 0x7f9d94004800 2024-04-28T12:37:20.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.977+0000 7f9db5400640 1 -- 172.21.15.131:0/254284079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d94004800 msgr2=0x7f9d94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.977+0000 7f9db5400640 1 --2- 172.21.15.131:0/254284079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d94004800 0x7f9d94004c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009ac0 tx=0x7f9d94005830 comp rx=0 tx=0).stop 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.977+0000 7f9daf200640 1 -- 172.21.15.131:0/254284079 reap_dead start 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.977+0000 7f9db5400640 1 -- 172.21.15.131:0/254284079 shutdown_connections 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.977+0000 7f9db5400640 1 -- 172.21.15.131:0/254284079 >> 172.21.15.131:0/254284079 conn(0x7f9db00b2890 msgr2=0x7f9db00b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.977+0000 7f9db5400640 1 -- 172.21.15.131:0/254284079 shutdown_connections 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.977+0000 7f9db5400640 1 -- 172.21.15.131:0/254284079 wait complete. 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.979+0000 7f9db5400640 1 Processor -- start 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.979+0000 7f9db5400640 1 -- start start 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.980+0000 7f9db5400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db013afb0 0x7f9db013de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.980+0000 7f9db5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9db013f6b0 0x7f9db013b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.980+0000 7f9db5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9db0157010 0x7f9db0157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.980+0000 7f9db5400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9db00b4f30 con 0x7f9db013afb0 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.980+0000 7f9db5400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9db00b4db0 con 0x7f9db013f6b0 2024-04-28T12:37:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.980+0000 7f9db5400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9db00cf7a0 con 0x7f9db0157010 2024-04-28T12:37:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.980+0000 7f9dae800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db013afb0 0x7f9db013de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.980+0000 7f9dae800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db013afb0 0x7f9db013de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55176/0 (socket says 172.21.15.131:55176) 2024-04-28T12:37:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.980+0000 7f9dae800640 1 -- 172.21.15.131:0/1473725607 learned_addr learned my addr 172.21.15.131:0/1473725607 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9da5e00640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9db013f6b0 0x7f9db013b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9dae800640 1 -- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9db0157010 msgr2=0x7f9db0157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9daf200640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9db0157010 0x7f9db0157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9dae800640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9db0157010 0x7f9db0157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9dae800640 1 -- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9db013f6b0 msgr2=0x7f9db013b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9dae800640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9db013f6b0 0x7f9db013b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9dae800640 1 -- 172.21.15.131:0/1473725607 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db013b8d0 con 0x7f9db013afb0 2024-04-28T12:37:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9da5e00640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9db013f6b0 0x7f9db013b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9dae800640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db013afb0 0x7f9db013de90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9da000db40 tx=0x7f9da0001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:20.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9da7400640 1 -- 172.21.15.131:0/1473725607 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0019070 con 0x7f9db013afb0 2024-04-28T12:37:20.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9da7400640 1 -- 172.21.15.131:0/1473725607 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da0004cb0 con 0x7f9db013afb0 2024-04-28T12:37:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9da7400640 1 -- 172.21.15.131:0/1473725607 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0004e60 con 0x7f9db013afb0 2024-04-28T12:37:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.981+0000 7f9db5400640 1 -- 172.21.15.131:0/1473725607 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db013fe20 con 0x7f9db013afb0 2024-04-28T12:37:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.982+0000 7f9db5400640 1 -- 172.21.15.131:0/1473725607 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db01403e0 con 0x7f9db013afb0 2024-04-28T12:37:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.983+0000 7f9db5400640 1 -- 172.21.15.131:0/1473725607 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db0069350 con 0x7f9db013afb0 2024-04-28T12:37:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.984+0000 7f9da7400640 1 -- 172.21.15.131:0/1473725607 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9da00072d0 con 0x7f9db013afb0 2024-04-28T12:37:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.985+0000 7f9da7400640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9d8c06c920 0x7f9d8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.985+0000 7f9da7400640 1 -- 172.21.15.131:0/1473725607 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f9da008b810 con 0x7f9db013afb0 2024-04-28T12:37:20.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.985+0000 7f9da5e00640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9d8c06c920 0x7f9d8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:20.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.986+0000 7f9da5e00640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9d8c06c920 0x7f9d8c06ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d98033000 tx=0x7f9d98008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:20.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:20.990+0000 7f9da7400640 1 -- 172.21.15.131:0/1473725607 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9da0059d80 con 0x7f9db013afb0 2024-04-28T12:37:21.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.191+0000 7f9db5400640 1 -- 172.21.15.131:0/1473725607 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f9db00b3f40 con 0x7f9db013afb0 2024-04-28T12:37:21.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.764+0000 7f9da7400640 1 -- 172.21.15.131:0/1473725607 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f9da0059680 con 0x7f9db013afb0 2024-04-28T12:37:21.761 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-04-28T12:37:21.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.768+0000 7f9db5400640 1 -- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9d8c06c920 msgr2=0x7f9d8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:21.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.768+0000 7f9db5400640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9d8c06c920 0x7f9d8c06ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d98033000 tx=0x7f9d98008030 comp rx=0 tx=0).stop 2024-04-28T12:37:21.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.768+0000 7f9db5400640 1 -- 172.21.15.131:0/1473725607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db013afb0 msgr2=0x7f9db013de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:21.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.768+0000 7f9db5400640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db013afb0 0x7f9db013de90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9da000db40 tx=0x7f9da0001700 comp rx=0 tx=0).stop 2024-04-28T12:37:21.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.768+0000 7f9db5400640 1 -- 172.21.15.131:0/1473725607 shutdown_connections 2024-04-28T12:37:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.768+0000 7f9db5400640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9d8c06c920 0x7f9d8c06ede0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.768+0000 7f9db5400640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9db0157010 0x7f9db0157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.768+0000 7f9db5400640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9db013f6b0 0x7f9db013b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.768+0000 7f9db5400640 1 --2- 172.21.15.131:0/1473725607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9db013afb0 0x7f9db013de90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.768+0000 7f9db5400640 1 -- 172.21.15.131:0/1473725607 >> 172.21.15.131:0/1473725607 conn(0x7f9db00b2890 msgr2=0x7f9db00b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.769+0000 7f9db5400640 1 -- 172.21.15.131:0/1473725607 shutdown_connections 2024-04-28T12:37:21.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.769+0000 7f9db5400640 1 -- 172.21.15.131:0/1473725607 wait complete. 2024-04-28T12:37:21.805 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-28T12:37:21.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.984+0000 7f016ec00640 1 Processor -- start 2024-04-28T12:37:21.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.984+0000 7f016ec00640 1 -- start start 2024-04-28T12:37:21.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.985+0000 7f016ec00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f016815b980 0x7f016806ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:21.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.985+0000 7f016ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01680b4d20 0x7f016806f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:21.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.985+0000 7f016ec00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f016815c2a0 con 0x7f016815bd60 2024-04-28T12:37:21.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.985+0000 7f016ec00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f016815d060 con 0x7f016815b980 2024-04-28T12:37:21.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.985+0000 7f016ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f016815c580 con 0x7f01680b4d20 2024-04-28T12:37:21.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.985+0000 7f016d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f016815b980 0x7f016806ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:21.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.986+0000 7f016d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f016815b980 0x7f016806ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55188/0 (socket says 172.21.15.131:55188) 2024-04-28T12:37:21.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.986+0000 7f016d800640 1 -- 172.21.15.131:0/2777126410 learned_addr learned my addr 172.21.15.131:0/2777126410 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:21.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.986+0000 7f016e200640 1 --2- 172.21.15.131:0/2777126410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01680b4d20 0x7f016806f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:21.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.986+0000 7f016d800640 1 -- 172.21.15.131:0/2777126410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01680b4d20 msgr2=0x7f016806f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:21.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.986+0000 7f016d800640 1 --2- 172.21.15.131:0/2777126410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01680b4d20 0x7f016806f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:21.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.986+0000 7f016d800640 1 -- 172.21.15.131:0/2777126410 >> v1:172.21.15.157:6789/0 conn(0x7f016815bd60 legacy=0x7f01680b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:21.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.986+0000 7f016d800640 1 -- 172.21.15.131:0/2777126410 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016815c860 con 0x7f016815b980 2024-04-28T12:37:21.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.987+0000 7f016d800640 1 --2- 172.21.15.131:0/2777126410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016815b980 0x7f016806ec90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f015c009870 tx=0x7f015c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf606102c383f913 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:21.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.987+0000 7f0167e00640 1 -- 172.21.15.131:0/2777126410 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c037070 con 0x7f016815b980 2024-04-28T12:37:21.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.987+0000 7f0167e00640 1 -- 172.21.15.131:0/2777126410 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c032440 con 0x7f016815b980 2024-04-28T12:37:21.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.987+0000 7f0167e00640 1 -- 172.21.15.131:0/2777126410 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c032810 con 0x7f016815b980 2024-04-28T12:37:21.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.988+0000 7f016ec00640 1 -- 172.21.15.131:0/2777126410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016815b980 msgr2=0x7f016806ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:21.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.988+0000 7f016ec00640 1 --2- 172.21.15.131:0/2777126410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016815b980 0x7f016806ec90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f015c009870 tx=0x7f015c02a210 comp rx=0 tx=0).stop 2024-04-28T12:37:21.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.988+0000 7f016ec00640 1 -- 172.21.15.131:0/2777126410 shutdown_connections 2024-04-28T12:37:21.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.988+0000 7f016ec00640 1 --2- 172.21.15.131:0/2777126410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01680b4d20 0x7f016806f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:21.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.988+0000 7f016ec00640 1 --2- 172.21.15.131:0/2777126410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016815b980 0x7f016806ec90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:21.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.988+0000 7f016ec00640 1 -- 172.21.15.131:0/2777126410 >> 172.21.15.131:0/2777126410 conn(0x7f0168068090 msgr2=0x7f01680684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:21.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.988+0000 7f016ec00640 1 -- 172.21.15.131:0/2777126410 shutdown_connections 2024-04-28T12:37:21.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.988+0000 7f016ec00640 1 -- 172.21.15.131:0/2777126410 wait complete. 2024-04-28T12:37:21.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.990+0000 7f016ec00640 1 Processor -- start 2024-04-28T12:37:21.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.990+0000 7f016ec00640 1 -- start start 2024-04-28T12:37:21.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.991+0000 7f016ec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01680b4d20 0x7f01681503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:21.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.991+0000 7f016ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f016815b980 0x7f01680cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:21.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.991+0000 7f016ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016815bd60 0x7f0168151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:21.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.991+0000 7f016ec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f016815f6b0 con 0x7f01680b4d20 2024-04-28T12:37:21.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.991+0000 7f016ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01680b52b0 con 0x7f016815bd60 2024-04-28T12:37:21.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.991+0000 7f016ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f016815f830 con 0x7f016815b980 2024-04-28T12:37:21.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.991+0000 7f016d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01680b4d20 0x7f01681503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:21.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.991+0000 7f016d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01680b4d20 0x7f01681503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55190/0 (socket says 172.21.15.131:55190) 2024-04-28T12:37:21.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.991+0000 7f016d800640 1 -- 172.21.15.131:0/485141127 learned_addr learned my addr 172.21.15.131:0/485141127 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:21.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.992+0000 7f016ce00640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f016815b980 0x7f01680cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:21.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.992+0000 7f016d800640 1 -- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f016815b980 msgr2=0x7f01680cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:21.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.992+0000 7f016d800640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f016815b980 0x7f01680cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:21.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.992+0000 7f016d800640 1 -- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016815bd60 msgr2=0x7f0168151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:21.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.992+0000 7f016d800640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016815bd60 0x7f0168151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:21.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.992+0000 7f016d800640 1 -- 172.21.15.131:0/485141127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016814f350 con 0x7f01680b4d20 2024-04-28T12:37:21.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.992+0000 7f016ce00640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f016815b980 0x7f01680cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:21.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.992+0000 7f016d800640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01680b4d20 0x7f01681503f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f015c009870 tx=0x7f015c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:21.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.992+0000 7f0165600640 1 -- 172.21.15.131:0/485141127 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c041020 con 0x7f01680b4d20 2024-04-28T12:37:21.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.992+0000 7f0165600640 1 -- 172.21.15.131:0/485141127 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c0306c0 con 0x7f01680b4d20 2024-04-28T12:37:21.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.992+0000 7f0165600640 1 -- 172.21.15.131:0/485141127 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c037040 con 0x7f01680b4d20 2024-04-28T12:37:21.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.992+0000 7f016ec00640 1 -- 172.21.15.131:0/485141127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016814f630 con 0x7f01680b4d20 2024-04-28T12:37:21.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.993+0000 7f016ec00640 1 -- 172.21.15.131:0/485141127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016814fbf0 con 0x7f01680b4d20 2024-04-28T12:37:21.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.993+0000 7f016ec00640 1 -- 172.21.15.131:0/485141127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01680506c0 con 0x7f01680b4d20 2024-04-28T12:37:21.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.995+0000 7f0165600640 1 -- 172.21.15.131:0/485141127 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f015c044020 con 0x7f01680b4d20 2024-04-28T12:37:21.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.996+0000 7f0165600640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f014806c920 0x7f014806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:21.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.996+0000 7f0165600640 1 -- 172.21.15.131:0/485141127 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f015c0ac2c0 con 0x7f01680b4d20 2024-04-28T12:37:21.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.996+0000 7f016ce00640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f014806c920 0x7f014806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:21.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:21.997+0000 7f016ce00640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f014806c920 0x7f014806ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f016814ea30 tx=0x7f0154008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:21.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.001+0000 7f0165600640 1 -- 172.21.15.131:0/485141127 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f015c04b030 con 0x7f01680b4d20 2024-04-28T12:37:22.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.234+0000 7f016ec00640 1 -- 172.21.15.131:0/485141127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f01680d0130 con 0x7f01680b4d20 2024-04-28T12:37:22.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.781+0000 7f0165600640 1 -- 172.21.15.131:0/485141127 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f015c07a740 con 0x7f01680b4d20 2024-04-28T12:37:22.784 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 2 and data pool 3 2024-04-28T12:37:22.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.789+0000 7f016ec00640 1 -- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f014806c920 msgr2=0x7f014806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:22.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.789+0000 7f016ec00640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f014806c920 0x7f014806ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f016814ea30 tx=0x7f0154008040 comp rx=0 tx=0).stop 2024-04-28T12:37:22.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.789+0000 7f016ec00640 1 -- 172.21.15.131:0/485141127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01680b4d20 msgr2=0x7f01681503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:22.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.789+0000 7f016ec00640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01680b4d20 0x7f01681503f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f015c009870 tx=0x7f015c0046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:22.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.790+0000 7f016ec00640 1 -- 172.21.15.131:0/485141127 shutdown_connections 2024-04-28T12:37:22.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.790+0000 7f016ec00640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f014806c920 0x7f014806ede0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:22.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.790+0000 7f016ec00640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f016815bd60 0x7f0168151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:22.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.790+0000 7f016ec00640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f016815b980 0x7f01680cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:22.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.790+0000 7f016ec00640 1 --2- 172.21.15.131:0/485141127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01680b4d20 0x7f01681503f0 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f015c009870 tx=0x7f015c0046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:22.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.790+0000 7f016ec00640 1 -- 172.21.15.131:0/485141127 >> 172.21.15.131:0/485141127 conn(0x7f0168068090 msgr2=0x7f01680d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:22.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.791+0000 7f016ec00640 1 -- 172.21.15.131:0/485141127 shutdown_connections 2024-04-28T12:37:22.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:22.792+0000 7f016ec00640 1 -- 172.21.15.131:0/485141127 wait complete. 2024-04-28T12:37:22.822 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-28T12:37:22.822 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-28T12:37:22.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.002+0000 7f0ad9000640 1 Processor -- start 2024-04-28T12:37:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.003+0000 7f0ad9000640 1 -- start start 2024-04-28T12:37:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.004+0000 7f0ad9000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0ad415f1a0 0x7f0ad40d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.004+0000 7f0ad9000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0ad40d2ce0 0x7f0ad40d3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.004+0000 7f0ad9000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ad40d0de0 con 0x7f0ad40d39b0 2024-04-28T12:37:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.004+0000 7f0ad9000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0ad40b8740 con 0x7f0ad40d2ce0 2024-04-28T12:37:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.004+0000 7f0ad9000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0ad40b7e10 con 0x7f0ad415f1a0 2024-04-28T12:37:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.004+0000 7f0ad1400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0ad40d2ce0 0x7f0ad40d3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.004+0000 7f0ad1400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0ad40d2ce0 0x7f0ad40d3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55196/0 (socket says 172.21.15.131:55196) 2024-04-28T12:37:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.004+0000 7f0ad1400640 1 -- 172.21.15.131:0/2170287110 learned_addr learned my addr 172.21.15.131:0/2170287110 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.005+0000 7f0ad1e00640 1 --2- 172.21.15.131:0/2170287110 >> v2:172.21.15.157:3300/0 conn(0x7f0ad415f1a0 0x7f0ad40d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.005+0000 7f0ad1400640 1 -- 172.21.15.131:0/2170287110 >> v2:172.21.15.157:3300/0 conn(0x7f0ad415f1a0 msgr2=0x7f0ad40d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.005+0000 7f0ad1400640 1 --2- 172.21.15.131:0/2170287110 >> v2:172.21.15.157:3300/0 conn(0x7f0ad415f1a0 0x7f0ad40d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.005+0000 7f0ad1400640 1 -- 172.21.15.131:0/2170287110 >> v1:172.21.15.157:6789/0 conn(0x7f0ad40d39b0 legacy=0x7f0ad40b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.005+0000 7f0ad1400640 1 -- 172.21.15.131:0/2170287110 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad40b80f0 con 0x7f0ad40d2ce0 2024-04-28T12:37:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.006+0000 7f0ad1400640 1 --2- 172.21.15.131:0/2170287110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad40d2ce0 0x7f0ad40d3470 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40d4e80 tx=0x7f0abc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18ea39350f28675 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:23.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.006+0000 7f0ad0a00640 1 -- 172.21.15.131:0/2170287110 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc037070 con 0x7f0ad40d2ce0 2024-04-28T12:37:23.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.006+0000 7f0ad0a00640 1 -- 172.21.15.131:0/2170287110 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0abc032440 con 0x7f0ad40d2ce0 2024-04-28T12:37:23.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.006+0000 7f0ad0a00640 1 -- 172.21.15.131:0/2170287110 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc032810 con 0x7f0ad40d2ce0 2024-04-28T12:37:23.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.007+0000 7f0ad9000640 1 -- 172.21.15.131:0/2170287110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad40d2ce0 msgr2=0x7f0ad40d3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.007+0000 7f0ad9000640 1 --2- 172.21.15.131:0/2170287110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad40d2ce0 0x7f0ad40d3470 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40d4e80 tx=0x7f0abc02a230 comp rx=0 tx=0).stop 2024-04-28T12:37:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.007+0000 7f0ad9000640 1 -- 172.21.15.131:0/2170287110 shutdown_connections 2024-04-28T12:37:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.007+0000 7f0ad9000640 1 --2- 172.21.15.131:0/2170287110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad40d2ce0 0x7f0ad40d3470 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.007+0000 7f0ad9000640 1 --2- 172.21.15.131:0/2170287110 >> v2:172.21.15.157:3300/0 conn(0x7f0ad415f1a0 0x7f0ad40d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.007+0000 7f0ad9000640 1 -- 172.21.15.131:0/2170287110 >> 172.21.15.131:0/2170287110 conn(0x7f0ad40d4f20 msgr2=0x7f0ad40d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.007+0000 7f0ad9000640 1 -- 172.21.15.131:0/2170287110 shutdown_connections 2024-04-28T12:37:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.007+0000 7f0ad9000640 1 -- 172.21.15.131:0/2170287110 wait complete. 2024-04-28T12:37:23.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.008+0000 7f0ad9000640 1 Processor -- start 2024-04-28T12:37:23.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.009+0000 7f0ad9000640 1 -- start start 2024-04-28T12:37:23.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.009+0000 7f0ad9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad40d2ce0 0x7f0ad40ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:23.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.009+0000 7f0ad9000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad40d39b0 0x7f0ad40a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:23.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad9000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad415f1a0 0x7f0ad40b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:23.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad9000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad415f580 con 0x7f0ad40d39b0 2024-04-28T12:37:23.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad9000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad40d0af0 con 0x7f0ad40d2ce0 2024-04-28T12:37:23.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad9000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad415f700 con 0x7f0ad415f1a0 2024-04-28T12:37:23.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad40d39b0 0x7f0ad40a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:23.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad40d39b0 0x7f0ad40a27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55204/0 (socket says 172.21.15.131:55204) 2024-04-28T12:37:23.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad1400640 1 -- 172.21.15.131:0/3164629800 learned_addr learned my addr 172.21.15.131:0/3164629800 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:23.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad1e00640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad40d2ce0 0x7f0ad40ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad2800640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad415f1a0 0x7f0ad40b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad1400640 1 -- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad415f1a0 msgr2=0x7f0ad40b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad1400640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad415f1a0 0x7f0ad40b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad1400640 1 -- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad40d2ce0 msgr2=0x7f0ad40ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad1400640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad40d2ce0 0x7f0ad40ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad1400640 1 -- 172.21.15.131:0/3164629800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad40ceb40 con 0x7f0ad40d39b0 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.010+0000 7f0ad2800640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad415f1a0 0x7f0ad40b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.011+0000 7f0ad1400640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad40d39b0 0x7f0ad40a27c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0abc000c00 tx=0x7f0abc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.011+0000 7f0aca000640 1 -- 172.21.15.131:0/3164629800 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc041070 con 0x7f0ad40d39b0 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.011+0000 7f0aca000640 1 -- 172.21.15.131:0/3164629800 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0abc03c870 con 0x7f0ad40d39b0 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.011+0000 7f0aca000640 1 -- 172.21.15.131:0/3164629800 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc037040 con 0x7f0ad40d39b0 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.011+0000 7f0ad9000640 1 -- 172.21.15.131:0/3164629800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad40cee20 con 0x7f0ad40d39b0 2024-04-28T12:37:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.011+0000 7f0ad9000640 1 -- 172.21.15.131:0/3164629800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad40b7410 con 0x7f0ad40d39b0 2024-04-28T12:37:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.012+0000 7f0ad9000640 1 -- 172.21.15.131:0/3164629800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad41f8680 con 0x7f0ad40d39b0 2024-04-28T12:37:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.013+0000 7f0aca000640 1 -- 172.21.15.131:0/3164629800 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0abc044070 con 0x7f0ad40d39b0 2024-04-28T12:37:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.014+0000 7f0aca000640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0ab006c920 0x7f0ab006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:23.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.014+0000 7f0aca000640 1 -- 172.21.15.131:0/3164629800 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f0abc0ac7b0 con 0x7f0ad40d39b0 2024-04-28T12:37:23.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.014+0000 7f0ad1e00640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0ab006c920 0x7f0ab006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.016+0000 7f0ad1e00640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0ab006c920 0x7f0ab006ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0acc006fd0 tx=0x7f0acc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:23.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.018+0000 7f0aca000640 1 -- 172.21.15.131:0/3164629800 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0abc07acf0 con 0x7f0ad40d39b0 2024-04-28T12:37:23.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.212+0000 7f0ad9000640 1 -- 172.21.15.131:0/3164629800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f0ad4069350 con 0x7f0ad40d39b0 2024-04-28T12:37:23.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.759+0000 7f0aca000640 1 -- 172.21.15.131:0/3164629800 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f0abc07a5f0 con 0x7f0ad40d39b0 2024-04-28T12:37:23.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.763+0000 7f0ad9000640 1 -- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0ab006c920 msgr2=0x7f0ab006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:23.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.763+0000 7f0ad9000640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0ab006c920 0x7f0ab006ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0acc006fd0 tx=0x7f0acc008040 comp rx=0 tx=0).stop 2024-04-28T12:37:23.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.763+0000 7f0ad9000640 1 -- 172.21.15.131:0/3164629800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad40d39b0 msgr2=0x7f0ad40a27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:23.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.763+0000 7f0ad9000640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad40d39b0 0x7f0ad40a27c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0abc000c00 tx=0x7f0abc0046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.764+0000 7f0ad9000640 1 -- 172.21.15.131:0/3164629800 shutdown_connections 2024-04-28T12:37:23.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.764+0000 7f0ad9000640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0ab006c920 0x7f0ab006ede0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.764+0000 7f0ad9000640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad415f1a0 0x7f0ad40b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.764+0000 7f0ad9000640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ad40d39b0 0x7f0ad40a27c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.764+0000 7f0ad9000640 1 --2- 172.21.15.131:0/3164629800 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad40d2ce0 0x7f0ad40ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.764+0000 7f0ad9000640 1 -- 172.21.15.131:0/3164629800 >> 172.21.15.131:0/3164629800 conn(0x7f0ad40d4f20 msgr2=0x7f0ad415d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:23.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.764+0000 7f0ad9000640 1 -- 172.21.15.131:0/3164629800 shutdown_connections 2024-04-28T12:37:23.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.764+0000 7f0ad9000640 1 -- 172.21.15.131:0/3164629800 wait complete. 2024-04-28T12:37:23.783 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-28T12:37:23.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.992+0000 7fa563e00640 1 Processor -- start 2024-04-28T12:37:23.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.993+0000 7fa563e00640 1 -- start start 2024-04-28T12:37:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.994+0000 7fa563e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa56415ee40 0x7fa5640d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.994+0000 7fa563e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa5640d35d0 0x7fa5640b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.994+0000 7fa563e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5640d04f0 con 0x7fa5640d57e0 2024-04-28T12:37:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.994+0000 7fa563e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa5640d02a0 con 0x7fa5640d35d0 2024-04-28T12:37:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.994+0000 7fa563e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa5640b4110 con 0x7fa56415ee40 2024-04-28T12:37:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.995+0000 7fa563400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa5640d35d0 0x7fa5640b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.995+0000 7fa563400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa5640d35d0 0x7fa5640b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55206/0 (socket says 172.21.15.131:55206) 2024-04-28T12:37:23.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.995+0000 7fa563400640 1 -- 172.21.15.131:0/281290016 learned_addr learned my addr 172.21.15.131:0/281290016 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:23.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.995+0000 7fa562a00640 1 --2- 172.21.15.131:0/281290016 >> v2:172.21.15.157:3300/0 conn(0x7fa56415ee40 0x7fa5640d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:23.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.995+0000 7fa563400640 1 -- 172.21.15.131:0/281290016 >> v2:172.21.15.157:3300/0 conn(0x7fa56415ee40 msgr2=0x7fa5640d2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:23.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.995+0000 7fa563400640 1 --2- 172.21.15.131:0/281290016 >> v2:172.21.15.157:3300/0 conn(0x7fa56415ee40 0x7fa5640d2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.995+0000 7fa563400640 1 -- 172.21.15.131:0/281290016 >> v1:172.21.15.157:6789/0 conn(0x7fa5640d57e0 legacy=0x7fa5640d2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:23.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.995+0000 7fa563400640 1 -- 172.21.15.131:0/281290016 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5640d07d0 con 0x7fa5640d35d0 2024-04-28T12:37:23.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.996+0000 7fa563400640 1 --2- 172.21.15.131:0/281290016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5640d35d0 0x7fa5640b4860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa56408d1a0 tx=0x7fa55c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1385a07d05f8569 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:23.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.997+0000 7fa561600640 1 -- 172.21.15.131:0/281290016 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c036070 con 0x7fa5640d35d0 2024-04-28T12:37:23.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.997+0000 7fa561600640 1 -- 172.21.15.131:0/281290016 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c031600 con 0x7fa5640d35d0 2024-04-28T12:37:23.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.997+0000 7fa561600640 1 -- 172.21.15.131:0/281290016 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0319d0 con 0x7fa5640d35d0 2024-04-28T12:37:23.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.998+0000 7fa563e00640 1 -- 172.21.15.131:0/281290016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5640d35d0 msgr2=0x7fa5640b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:23.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.998+0000 7fa563e00640 1 --2- 172.21.15.131:0/281290016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5640d35d0 0x7fa5640b4860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa56408d1a0 tx=0x7fa55c02a6f0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.998+0000 7fa563e00640 1 -- 172.21.15.131:0/281290016 shutdown_connections 2024-04-28T12:37:23.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.998+0000 7fa563e00640 1 --2- 172.21.15.131:0/281290016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5640d35d0 0x7fa5640b4860 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.998+0000 7fa563e00640 1 --2- 172.21.15.131:0/281290016 >> v2:172.21.15.157:3300/0 conn(0x7fa56415ee40 0x7fa5640d2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:23.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.998+0000 7fa563e00640 1 -- 172.21.15.131:0/281290016 >> 172.21.15.131:0/281290016 conn(0x7fa5640d4cc0 msgr2=0x7fa5640d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:23.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.998+0000 7fa563e00640 1 -- 172.21.15.131:0/281290016 shutdown_connections 2024-04-28T12:37:23.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:23.998+0000 7fa563e00640 1 -- 172.21.15.131:0/281290016 wait complete. 2024-04-28T12:37:23.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.000+0000 7fa563e00640 1 Processor -- start 2024-04-28T12:37:23.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.000+0000 7fa563e00640 1 -- start start 2024-04-28T12:37:24.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa563e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5640d35d0 0x7fa564161670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:24.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa563e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5640d57e0 0x7fa564160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:24.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa563e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa56415ee40 0x7fa5640ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:24.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa563e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa56415f220 con 0x7fa5640d35d0 2024-04-28T12:37:24.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa563e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5640b4360 con 0x7fa56415ee40 2024-04-28T12:37:24.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa563e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa56415f3a0 con 0x7fa5640d57e0 2024-04-28T12:37:24.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa562a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5640d35d0 0x7fa564161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:24.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa562a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5640d35d0 0x7fa564161670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55210/0 (socket says 172.21.15.131:55210) 2024-04-28T12:37:24.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa562a00640 1 -- 172.21.15.131:0/793448294 learned_addr learned my addr 172.21.15.131:0/793448294 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:24.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa562a00640 1 -- 172.21.15.131:0/793448294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5640d57e0 msgr2=0x7fa564160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:24.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa562a00640 1 --2- 172.21.15.131:0/793448294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5640d57e0 0x7fa564160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:24.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa562a00640 1 -- 172.21.15.131:0/793448294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa56415ee40 msgr2=0x7fa5640ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:24.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa562a00640 1 --2- 172.21.15.131:0/793448294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa56415ee40 0x7fa5640ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:24.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.001+0000 7fa562a00640 1 -- 172.21.15.131:0/793448294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5640b87e0 con 0x7fa5640d35d0 2024-04-28T12:37:24.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.002+0000 7fa562a00640 1 --2- 172.21.15.131:0/793448294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5640d35d0 0x7fa564161670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa55400d830 tx=0x7fa55400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:24.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.002+0000 7fa55b400640 1 -- 172.21.15.131:0/793448294 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55400b6c0 con 0x7fa5640d35d0 2024-04-28T12:37:24.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.002+0000 7fa55b400640 1 -- 172.21.15.131:0/793448294 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa554019070 con 0x7fa5640d35d0 2024-04-28T12:37:24.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.002+0000 7fa55b400640 1 -- 172.21.15.131:0/793448294 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55400ba80 con 0x7fa5640d35d0 2024-04-28T12:37:24.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.002+0000 7fa563e00640 1 -- 172.21.15.131:0/793448294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5640b8a80 con 0x7fa5640d35d0 2024-04-28T12:37:24.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.002+0000 7fa563e00640 1 -- 172.21.15.131:0/793448294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5640b9040 con 0x7fa5640d35d0 2024-04-28T12:37:24.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.003+0000 7fa563e00640 1 -- 172.21.15.131:0/793448294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5640b5340 con 0x7fa5640d35d0 2024-04-28T12:37:24.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.004+0000 7fa55b400640 1 -- 172.21.15.131:0/793448294 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa55401e460 con 0x7fa5640d35d0 2024-04-28T12:37:24.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.005+0000 7fa55b400640 1 --2- 172.21.15.131:0/793448294 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa54406c8f0 0x7fa54406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:24.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.005+0000 7fa55b400640 1 -- 172.21.15.131:0/793448294 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7fa55408c300 con 0x7fa5640d35d0 2024-04-28T12:37:24.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.006+0000 7fa562000640 1 --2- 172.21.15.131:0/793448294 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa54406c8f0 0x7fa54406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:24.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.007+0000 7fa562000640 1 --2- 172.21.15.131:0/793448294 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa54406c8f0 0x7fa54406edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa564162880 tx=0x7fa5500079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:24.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.010+0000 7fa55b400640 1 -- 172.21.15.131:0/793448294 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa5640b5340 con 0x7fa5640d35d0 2024-04-28T12:37:24.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:24.211+0000 7fa563e00640 1 -- 172.21.15.131:0/793448294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa5640506c0 con 0x7fa5640d35d0 2024-04-28T12:37:25.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.817+0000 7fa55b400640 1 -- 172.21.15.131:0/793448294 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7fa55405a690 con 0x7fa5640d35d0 2024-04-28T12:37:25.814 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data_ec' created 2024-04-28T12:37:25.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.821+0000 7fa563e00640 1 -- 172.21.15.131:0/793448294 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa54406c8f0 msgr2=0x7fa54406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:25.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.821+0000 7fa563e00640 1 --2- 172.21.15.131:0/793448294 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa54406c8f0 0x7fa54406edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa564162880 tx=0x7fa5500079b0 comp rx=0 tx=0).stop 2024-04-28T12:37:25.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.821+0000 7fa563e00640 1 -- 172.21.15.131:0/793448294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5640d35d0 msgr2=0x7fa564161670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:25.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.821+0000 7fa563e00640 1 --2- 172.21.15.131:0/793448294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5640d35d0 0x7fa564161670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa55400d830 tx=0x7fa55400dcf0 comp rx=0 tx=0).stop 2024-04-28T12:37:25.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.822+0000 7fa563e00640 1 -- 172.21.15.131:0/793448294 shutdown_connections 2024-04-28T12:37:25.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.822+0000 7fa563e00640 1 --2- 172.21.15.131:0/793448294 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa54406c8f0 0x7fa54406edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:25.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.822+0000 7fa563e00640 1 --2- 172.21.15.131:0/793448294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa56415ee40 0x7fa5640ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:25.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.822+0000 7fa563e00640 1 --2- 172.21.15.131:0/793448294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5640d57e0 0x7fa564160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:25.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.822+0000 7fa563e00640 1 --2- 172.21.15.131:0/793448294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5640d35d0 0x7fa564161670 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:25.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.822+0000 7fa563e00640 1 -- 172.21.15.131:0/793448294 >> 172.21.15.131:0/793448294 conn(0x7fa5640d4cc0 msgr2=0x7fa56406e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:25.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.822+0000 7fa563e00640 1 -- 172.21.15.131:0/793448294 shutdown_connections 2024-04-28T12:37:25.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:25.823+0000 7fa563e00640 1 -- 172.21.15.131:0/793448294 wait complete. 2024-04-28T12:37:25.849 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-28T12:37:25.989 INFO:tasks.ceph.mds.b.smithi157.stderr:2024-04-28T12:37:25.992+0000 7f24c0600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-28T12:37:26.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.031+0000 7fc4e5000640 1 Processor -- start 2024-04-28T12:37:26.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.032+0000 7fc4e5000640 1 -- start start 2024-04-28T12:37:26.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.033+0000 7fc4e5000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4e006e5a0 0x7fc4e0163a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:26.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.033+0000 7fc4e5000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e0073740 con 0x7fc4e0072a80 2024-04-28T12:37:26.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.033+0000 7fc4e5000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e00b3a90 con 0x7fc4e014cd60 2024-04-28T12:37:26.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.033+0000 7fc4e5000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e00b3d70 con 0x7fc4e006e5a0 2024-04-28T12:37:26.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.033+0000 7fc4de800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc4e0072a80 0x7fc4e0072e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54884/0 (socket says 172.21.15.131:54884) 2024-04-28T12:37:26.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.033+0000 7fc4de800640 1 -- 172.21.15.131:0/2964985188 learned_addr learned my addr 172.21.15.131:0/2964985188 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:26.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4d5400640 1 --2- 172.21.15.131:0/2964985188 >> v2:172.21.15.157:3300/0 conn(0x7fc4e006e5a0 0x7fc4e0163a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:26.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 753871758 0 0) 0x7fc4e0073740 con 0x7fc4e0072a80 2024-04-28T12:37:26.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4c4003520 con 0x7fc4e0072a80 2024-04-28T12:37:26.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 924329622 0 0) 0x7fc4c4003520 con 0x7fc4e0072a80 2024-04-28T12:37:26.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 >> v2:172.21.15.157:3300/0 conn(0x7fc4e006e5a0 msgr2=0x7fc4e0163a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:26.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4dd400640 1 --2- 172.21.15.131:0/2964985188 >> v2:172.21.15.157:3300/0 conn(0x7fc4e006e5a0 0x7fc4e0163a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:26.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 >> v1:172.21.15.157:6789/0 conn(0x7fc4e014cd60 legacy=0x7fc4e006de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e00b4050 con 0x7fc4e0072a80 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fc4d80019f0 con 0x7fc4e0072a80 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 >> v1:172.21.15.131:6789/0 conn(0x7fc4e0072a80 legacy=0x7fc4e0072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4dd400640 1 --2- 172.21.15.131:0/2964985188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c40039b0 0x7fc4c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4dd400640 1 --2- 172.21.15.131:0/2964985188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c4004800 0x7fc4c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.034+0000 7fc4dd400640 1 --2- 172.21.15.131:0/2964985188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c40051c0 0x7fc4c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.035+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e00b3d70 con 0x7fc4c40051c0 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.035+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c400c060 con 0x7fc4c4004800 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.035+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c400c260 con 0x7fc4c40039b0 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.035+0000 7fc4de800640 1 --2- 172.21.15.131:0/2964985188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c40051c0 0x7fc4c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.035+0000 7fc4dde00640 1 --2- 172.21.15.131:0/2964985188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c40039b0 0x7fc4c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.035+0000 7fc4d5400640 1 --2- 172.21.15.131:0/2964985188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c4004800 0x7fc4c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.035+0000 7fc4de800640 1 -- 172.21.15.131:0/2964985188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c40039b0 msgr2=0x7fc4c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:26.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.035+0000 7fc4de800640 1 --2- 172.21.15.131:0/2964985188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c40039b0 0x7fc4c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:26.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.035+0000 7fc4de800640 1 -- 172.21.15.131:0/2964985188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c4004800 msgr2=0x7fc4c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:26.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.035+0000 7fc4de800640 1 --2- 172.21.15.131:0/2964985188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4c4004800 0x7fc4c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:26.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.035+0000 7fc4de800640 1 -- 172.21.15.131:0/2964985188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c4003520 con 0x7fc4c40051c0 2024-04-28T12:37:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.036+0000 7fc4de800640 1 -- 172.21.15.131:0/2964985188 reap_dead start 2024-04-28T12:37:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.036+0000 7fc4de800640 1 --2- 172.21.15.131:0/2964985188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c40051c0 0x7fc4c400b9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800a860 tx=0x7fc4c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6b4ff14af11382 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.036+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800a480 con 0x7fc4c40051c0 2024-04-28T12:37:26.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.036+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d802f070 con 0x7fc4c40051c0 2024-04-28T12:37:26.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.036+0000 7fc4dd400640 1 -- 172.21.15.131:0/2964985188 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d802bca0 con 0x7fc4c40051c0 2024-04-28T12:37:26.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.037+0000 7fc4e5000640 1 -- 172.21.15.131:0/2964985188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c40051c0 msgr2=0x7fc4c400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:26.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.037+0000 7fc4e5000640 1 --2- 172.21.15.131:0/2964985188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c40051c0 0x7fc4c400b9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800a860 tx=0x7fc4c4005830 comp rx=0 tx=0).stop 2024-04-28T12:37:26.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.037+0000 7fc4e5000640 1 -- 172.21.15.131:0/2964985188 shutdown_connections 2024-04-28T12:37:26.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.037+0000 7fc4e5000640 1 --2- 172.21.15.131:0/2964985188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4c40051c0 0x7fc4c400b9c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:26.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.037+0000 7fc4e5000640 1 -- 172.21.15.131:0/2964985188 >> 172.21.15.131:0/2964985188 conn(0x7fc4e00b2790 msgr2=0x7fc4e00b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:26.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.037+0000 7fc4e5000640 1 -- 172.21.15.131:0/2964985188 shutdown_connections 2024-04-28T12:37:26.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.037+0000 7fc4e5000640 1 -- 172.21.15.131:0/2964985188 wait complete. 2024-04-28T12:37:26.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.039+0000 7fc4e5000640 1 Processor -- start 2024-04-28T12:37:26.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.039+0000 7fc4e5000640 1 -- start start 2024-04-28T12:37:26.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.040+0000 7fc4e5000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c40051c0 0x7fc4e01f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:26.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.040+0000 7fc4e5000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e0144540 0x7fc4e01449a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:26.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.040+0000 7fc4e5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e0146e40 0x7fc4e01472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:26.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.040+0000 7fc4e5000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e014d2c0 con 0x7fc4e0146e40 2024-04-28T12:37:26.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.040+0000 7fc4e5000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e014d140 con 0x7fc4e0144540 2024-04-28T12:37:26.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.040+0000 7fc4e5000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e0163a90 con 0x7fc4c40051c0 2024-04-28T12:37:26.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.040+0000 7fc4de800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e0146e40 0x7fc4e01472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.040+0000 7fc4de800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e0146e40 0x7fc4e01472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55238/0 (socket says 172.21.15.131:55238) 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.040+0000 7fc4de800640 1 -- 172.21.15.131:0/509809601 learned_addr learned my addr 172.21.15.131:0/509809601 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.040+0000 7fc4dde00640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c40051c0 0x7fc4e01f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.040+0000 7fc4d5400640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e0144540 0x7fc4e01449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.041+0000 7fc4de800640 1 -- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c40051c0 msgr2=0x7fc4e01f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.041+0000 7fc4de800640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c40051c0 0x7fc4e01f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.041+0000 7fc4de800640 1 -- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e0144540 msgr2=0x7fc4e01449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.041+0000 7fc4de800640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e0144540 0x7fc4e01449a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.041+0000 7fc4de800640 1 -- 172.21.15.131:0/509809601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e0142b00 con 0x7fc4e0146e40 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.041+0000 7fc4de800640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e0146e40 0x7fc4e01472f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800a980 tx=0x7fc4d802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.041+0000 7fc4d6a00640 1 -- 172.21.15.131:0/509809601 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800a480 con 0x7fc4e0146e40 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.041+0000 7fc4d6a00640 1 -- 172.21.15.131:0/509809601 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d8034670 con 0x7fc4e0146e40 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.041+0000 7fc4d6a00640 1 -- 172.21.15.131:0/509809601 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8034a40 con 0x7fc4e0146e40 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.041+0000 7fc4e5000640 1 -- 172.21.15.131:0/509809601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e0145090 con 0x7fc4e0146e40 2024-04-28T12:37:26.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.041+0000 7fc4e5000640 1 -- 172.21.15.131:0/509809601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e0145650 con 0x7fc4e0146e40 2024-04-28T12:37:26.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.042+0000 7fc4e5000640 1 -- 172.21.15.131:0/509809601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e0069350 con 0x7fc4e0146e40 2024-04-28T12:37:26.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.044+0000 7fc4d6a00640 1 -- 172.21.15.131:0/509809601 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4d8034be0 con 0x7fc4e0146e40 2024-04-28T12:37:26.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.044+0000 7fc4d6a00640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4bc06c920 0x7fc4bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:26.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.045+0000 7fc4d6a00640 1 -- 172.21.15.131:0/509809601 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7fc4d80abc20 con 0x7fc4e0146e40 2024-04-28T12:37:26.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.045+0000 7fc4dde00640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4bc06c920 0x7fc4bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:26.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.046+0000 7fc4dde00640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4bc06c920 0x7fc4bc06ede0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0000c00 tx=0x7fc4d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:26.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.050+0000 7fc4d6a00640 1 -- 172.21.15.131:0/509809601 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4d8046060 con 0x7fc4e0146e40 2024-04-28T12:37:26.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.248+0000 7fc4e5000640 1 -- 172.21.15.131:0/509809601 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fc4e006e0a0 con 0x7fc4e0146e40 2024-04-28T12:37:26.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.830+0000 7fc4d6a00640 1 -- 172.21.15.131:0/509809601 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7fc4d80ab070 con 0x7fc4e0146e40 2024-04-28T12:37:26.827 INFO:teuthology.orchestra.run.smithi131.stderr:set pool 4 allow_ec_overwrites to true 2024-04-28T12:37:26.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.834+0000 7fc4e5000640 1 -- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4bc06c920 msgr2=0x7fc4bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:26.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.834+0000 7fc4e5000640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4bc06c920 0x7fc4bc06ede0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0000c00 tx=0x7fc4d0009040 comp rx=0 tx=0).stop 2024-04-28T12:37:26.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.834+0000 7fc4e5000640 1 -- 172.21.15.131:0/509809601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e0146e40 msgr2=0x7fc4e01472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:26.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.834+0000 7fc4e5000640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e0146e40 0x7fc4e01472f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800a980 tx=0x7fc4d802d070 comp rx=0 tx=0).stop 2024-04-28T12:37:26.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.835+0000 7fc4e5000640 1 -- 172.21.15.131:0/509809601 shutdown_connections 2024-04-28T12:37:26.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.835+0000 7fc4e5000640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4bc06c920 0x7fc4bc06ede0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:26.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.835+0000 7fc4e5000640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4e0146e40 0x7fc4e01472f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:26.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.835+0000 7fc4e5000640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e0144540 0x7fc4e01449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:26.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.835+0000 7fc4e5000640 1 --2- 172.21.15.131:0/509809601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4c40051c0 0x7fc4e01f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:26.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.835+0000 7fc4e5000640 1 -- 172.21.15.131:0/509809601 >> 172.21.15.131:0/509809601 conn(0x7fc4e00b2790 msgr2=0x7fc4e014bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:26.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.836+0000 7fc4e5000640 1 -- 172.21.15.131:0/509809601 shutdown_connections 2024-04-28T12:37:26.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:26.836+0000 7fc4e5000640 1 -- 172.21.15.131:0/509809601 wait complete. 2024-04-28T12:37:26.868 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-28T12:37:27.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.052+0000 7fb2d5800640 1 Processor -- start 2024-04-28T12:37:27.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.052+0000 7fb2d5800640 1 -- start start 2024-04-28T12:37:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.054+0000 7fb2d5800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d015b980 0x7fb2d006ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.054+0000 7fb2d5800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb2d015bd60 0x7fb2d006f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.054+0000 7fb2d5800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d015c2a0 con 0x7fb2d00b4610 2024-04-28T12:37:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.054+0000 7fb2d5800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb2d015d060 con 0x7fb2d015bd60 2024-04-28T12:37:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.054+0000 7fb2d5800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d015c580 con 0x7fb2d015b980 2024-04-28T12:37:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.054+0000 7fb2c7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb2d015bd60 0x7fb2d006f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.054+0000 7fb2c7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb2d015bd60 0x7fb2d006f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55244/0 (socket says 172.21.15.131:55244) 2024-04-28T12:37:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.054+0000 7fb2c7400640 1 -- 172.21.15.131:0/1424281561 learned_addr learned my addr 172.21.15.131:0/1424281561 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:27.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.054+0000 7fb2d4e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb2d00b4610 0x7fb2d00b4aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54886/0 (socket says 172.21.15.131:54886) 2024-04-28T12:37:27.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.055+0000 7fb2cfe00640 1 --2- 172.21.15.131:0/1424281561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d015b980 0x7fb2d006ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:27.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.055+0000 7fb2c7400640 1 -- 172.21.15.131:0/1424281561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d015b980 msgr2=0x7fb2d006ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:27.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.055+0000 7fb2c7400640 1 --2- 172.21.15.131:0/1424281561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d015b980 0x7fb2d006ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:27.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.055+0000 7fb2c7400640 1 -- 172.21.15.131:0/1424281561 >> v1:172.21.15.131:6789/0 conn(0x7fb2d00b4610 legacy=0x7fb2d00b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:27.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.055+0000 7fb2c7400640 1 -- 172.21.15.131:0/1424281561 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d015c860 con 0x7fb2d015bd60 2024-04-28T12:37:27.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.055+0000 7fb2cf400640 1 -- 172.21.15.131:0/1424281561 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 386483529 0 0) 0x7fb2d015c2a0 con 0x7fb2d00b4610 2024-04-28T12:37:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.056+0000 7fb2c7400640 1 --2- 172.21.15.131:0/1424281561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d015bd60 0x7fb2d006f1d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800bb10 tx=0x7fb2b802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadb184420c3e4c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.056+0000 7fb2cf400640 1 -- 172.21.15.131:0/1424281561 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8036020 con 0x7fb2d015bd60 2024-04-28T12:37:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.056+0000 7fb2cf400640 1 -- 172.21.15.131:0/1424281561 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b8032820 con 0x7fb2d015bd60 2024-04-28T12:37:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.056+0000 7fb2cf400640 1 -- 172.21.15.131:0/1424281561 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8032bf0 con 0x7fb2d015bd60 2024-04-28T12:37:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.057+0000 7fb2d5800640 1 -- 172.21.15.131:0/1424281561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d015bd60 msgr2=0x7fb2d006f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.057+0000 7fb2d5800640 1 --2- 172.21.15.131:0/1424281561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d015bd60 0x7fb2d006f1d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800bb10 tx=0x7fb2b802c920 comp rx=0 tx=0).stop 2024-04-28T12:37:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.057+0000 7fb2d5800640 1 -- 172.21.15.131:0/1424281561 shutdown_connections 2024-04-28T12:37:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.057+0000 7fb2d5800640 1 --2- 172.21.15.131:0/1424281561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d015bd60 0x7fb2d006f1d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.057+0000 7fb2d5800640 1 --2- 172.21.15.131:0/1424281561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d015b980 0x7fb2d006ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.057+0000 7fb2d5800640 1 -- 172.21.15.131:0/1424281561 >> 172.21.15.131:0/1424281561 conn(0x7fb2d0068090 msgr2=0x7fb2d00684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:27.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.057+0000 7fb2d5800640 1 -- 172.21.15.131:0/1424281561 shutdown_connections 2024-04-28T12:37:27.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.057+0000 7fb2d5800640 1 -- 172.21.15.131:0/1424281561 wait complete. 2024-04-28T12:37:27.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.059+0000 7fb2d5800640 1 Processor -- start 2024-04-28T12:37:27.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.059+0000 7fb2d5800640 1 -- start start 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2d5800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d00b4610 0x7fb2d00d0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2d5800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d015b980 0x7fb2d01f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2d5800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d015bd60 0x7fb2d01544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2d5800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d006db80 con 0x7fb2d00b4610 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2d5800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d00b52b0 con 0x7fb2d015bd60 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2d5800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d006dd00 con 0x7fb2d015b980 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2cfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d00b4610 0x7fb2d00d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2cfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d00b4610 0x7fb2d00d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55258/0 (socket says 172.21.15.131:55258) 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2cfe00640 1 -- 172.21.15.131:0/514959393 learned_addr learned my addr 172.21.15.131:0/514959393 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2c7400640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d015b980 0x7fb2d01f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2d4e00640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d015bd60 0x7fb2d01544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2cfe00640 1 -- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d015b980 msgr2=0x7fb2d01f08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:27.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2cfe00640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d015b980 0x7fb2d01f08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:27.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2cfe00640 1 -- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d015bd60 msgr2=0x7fb2d01544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:27.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2cfe00640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d015bd60 0x7fb2d01544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:27.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2cfe00640 1 -- 172.21.15.131:0/514959393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d00b1820 con 0x7fb2d00b4610 2024-04-28T12:37:27.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2c7400640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d015b980 0x7fb2d01f08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:27.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.061+0000 7fb2d4e00640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d015bd60 0x7fb2d01544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:27.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.062+0000 7fb2cfe00640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d00b4610 0x7fb2d00d0660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0007ef0 tx=0x7fb2c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:27.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.062+0000 7fb2ccc00640 1 -- 172.21.15.131:0/514959393 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0014620 con 0x7fb2d00b4610 2024-04-28T12:37:27.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.062+0000 7fb2ccc00640 1 -- 172.21.15.131:0/514959393 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c0019070 con 0x7fb2d00b4610 2024-04-28T12:37:27.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.062+0000 7fb2ccc00640 1 -- 172.21.15.131:0/514959393 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0014a00 con 0x7fb2d00b4610 2024-04-28T12:37:27.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.062+0000 7fb2d5800640 1 -- 172.21.15.131:0/514959393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d00b1b00 con 0x7fb2d00b4610 2024-04-28T12:37:27.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.062+0000 7fb2d5800640 1 -- 172.21.15.131:0/514959393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d00b20c0 con 0x7fb2d00b4610 2024-04-28T12:37:27.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.064+0000 7fb2d5800640 1 -- 172.21.15.131:0/514959393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d00506c0 con 0x7fb2d00b4610 2024-04-28T12:37:27.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.064+0000 7fb2ccc00640 1 -- 172.21.15.131:0/514959393 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2c0014ba0 con 0x7fb2d00b4610 2024-04-28T12:37:27.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.065+0000 7fb2ccc00640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb2ac06c8f0 0x7fb2ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:27.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.065+0000 7fb2ccc00640 1 -- 172.21.15.131:0/514959393 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fb2c008c7b0 con 0x7fb2d00b4610 2024-04-28T12:37:27.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.066+0000 7fb2c7400640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb2ac06c8f0 0x7fb2ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:27.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.067+0000 7fb2c7400640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb2ac06c8f0 0x7fb2ac06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8037000 tx=0x7fb2b803d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:27.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.070+0000 7fb2ccc00640 1 -- 172.21.15.131:0/514959393 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb2c005a8c0 con 0x7fb2d00b4610 2024-04-28T12:37:27.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.315+0000 7fb2d5800640 1 -- 172.21.15.131:0/514959393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fb2d015f1c0 con 0x7fb2d00b4610 2024-04-28T12:37:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.845+0000 7fb2ccc00640 1 -- 172.21.15.131:0/514959393 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fb2c005a1c0 con 0x7fb2d00b4610 2024-04-28T12:37:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:added data pool 4 to fsmap 2024-04-28T12:37:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.848+0000 7fb2d5800640 1 -- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb2ac06c8f0 msgr2=0x7fb2ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.848+0000 7fb2d5800640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb2ac06c8f0 0x7fb2ac06edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8037000 tx=0x7fb2b803d040 comp rx=0 tx=0).stop 2024-04-28T12:37:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.849+0000 7fb2d5800640 1 -- 172.21.15.131:0/514959393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d00b4610 msgr2=0x7fb2d00d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.849+0000 7fb2d5800640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d00b4610 0x7fb2d00d0660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0007ef0 tx=0x7fb2c000e4b0 comp rx=0 tx=0).stop 2024-04-28T12:37:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.849+0000 7fb2d5800640 1 -- 172.21.15.131:0/514959393 shutdown_connections 2024-04-28T12:37:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.849+0000 7fb2d5800640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb2ac06c8f0 0x7fb2ac06edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.849+0000 7fb2d5800640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2d015bd60 0x7fb2d01544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.849+0000 7fb2d5800640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2d015b980 0x7fb2d01f08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.849+0000 7fb2d5800640 1 --2- 172.21.15.131:0/514959393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2d00b4610 0x7fb2d00d0660 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.849+0000 7fb2d5800640 1 -- 172.21.15.131:0/514959393 >> 172.21.15.131:0/514959393 conn(0x7fb2d0068090 msgr2=0x7fb2d0157580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.849+0000 7fb2d5800640 1 -- 172.21.15.131:0/514959393 shutdown_connections 2024-04-28T12:37:27.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:27.849+0000 7fb2d5800640 1 -- 172.21.15.131:0/514959393 wait complete. 2024-04-28T12:37:27.867 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:37:28.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.070+0000 7f1f46000640 1 Processor -- start 2024-04-28T12:37:28.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.071+0000 7f1f46000640 1 -- start start 2024-04-28T12:37:28.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.072+0000 7f1f46000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400cd180 0x7f1f40072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.072+0000 7f1f46000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f400cd560 con 0x7f1f4015dc50 2024-04-28T12:37:28.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.072+0000 7f1f46000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f40072d00 con 0x7f1f400ce480 2024-04-28T12:37:28.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.072+0000 7f1f46000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1f40072fe0 con 0x7f1f400cd180 2024-04-28T12:37:28.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.072+0000 7f1f3fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1f4015dc50 0x7f1f400cdd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54902/0 (socket says 172.21.15.131:54902) 2024-04-28T12:37:28.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.072+0000 7f1f3fe00640 1 -- 172.21.15.131:0/1246050266 learned_addr learned my addr 172.21.15.131:0/1246050266 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:28.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f44c00640 1 --2- 172.21.15.131:0/1246050266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400cd180 0x7f1f40072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3173103854 0 0) 0x7f1f400cd560 con 0x7f1f4015dc50 2024-04-28T12:37:28.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f28003520 con 0x7f1f4015dc50 2024-04-28T12:37:28.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3463517274 0 0) 0x7f1f28003520 con 0x7f1f4015dc50 2024-04-28T12:37:28.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400cd180 msgr2=0x7f1f40072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:28.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 --2- 172.21.15.131:0/1246050266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400cd180 0x7f1f40072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 >> v1:172.21.15.157:6789/0 conn(0x7f1f400ce480 legacy=0x7f1f4006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f400732c0 con 0x7f1f4015dc50 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f1f2c001670 con 0x7f1f4015dc50 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 >> v1:172.21.15.131:6789/0 conn(0x7f1f4015dc50 legacy=0x7f1f400cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 --2- 172.21.15.131:0/1246050266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f280039e0 0x7f1f28003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 --2- 172.21.15.131:0/1246050266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400cd180 0x7f1f28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 --2- 172.21.15.131:0/1246050266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f28004d40 0x7f1f280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1f40072fe0 con 0x7f1f28004d40 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2800baa0 con 0x7f1f280039e0 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.073+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2800bca0 con 0x7f1f400cd180 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.074+0000 7f1f45600640 1 --2- 172.21.15.131:0/1246050266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f28004d40 0x7f1f280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.074+0000 7f1f44c00640 1 --2- 172.21.15.131:0/1246050266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f280039e0 0x7f1f28003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.074+0000 7f1f3fe00640 1 --2- 172.21.15.131:0/1246050266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400cd180 0x7f1f28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.074+0000 7f1f45600640 1 -- 172.21.15.131:0/1246050266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400cd180 msgr2=0x7f1f28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:28.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.074+0000 7f1f45600640 1 --2- 172.21.15.131:0/1246050266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400cd180 0x7f1f28004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.074+0000 7f1f45600640 1 -- 172.21.15.131:0/1246050266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f280039e0 msgr2=0x7f1f28003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:28.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.074+0000 7f1f45600640 1 --2- 172.21.15.131:0/1246050266 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f280039e0 0x7f1f28003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.074+0000 7f1f45600640 1 -- 172.21.15.131:0/1246050266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f28003520 con 0x7f1f28004d40 2024-04-28T12:37:28.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.075+0000 7f1f45600640 1 --2- 172.21.15.131:0/1246050266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f28004d40 0x7f1f280051c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1f40071a00 tx=0x7f1f28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb46f0c60fd4746 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:28.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.075+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38001a70 con 0x7f1f28004d40 2024-04-28T12:37:28.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.075+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f3802add0 con 0x7f1f28004d40 2024-04-28T12:37:28.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.075+0000 7f1f3f400640 1 -- 172.21.15.131:0/1246050266 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38003bb0 con 0x7f1f28004d40 2024-04-28T12:37:28.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.076+0000 7f1f46000640 1 -- 172.21.15.131:0/1246050266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f28004d40 msgr2=0x7f1f280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.076+0000 7f1f46000640 1 --2- 172.21.15.131:0/1246050266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f28004d40 0x7f1f280051c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1f40071a00 tx=0x7f1f28005830 comp rx=0 tx=0).stop 2024-04-28T12:37:28.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.076+0000 7f1f45600640 1 -- 172.21.15.131:0/1246050266 reap_dead start 2024-04-28T12:37:28.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.076+0000 7f1f46000640 1 -- 172.21.15.131:0/1246050266 shutdown_connections 2024-04-28T12:37:28.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.076+0000 7f1f46000640 1 -- 172.21.15.131:0/1246050266 >> 172.21.15.131:0/1246050266 conn(0x7f1f40068090 msgr2=0x7f1f40068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:28.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.076+0000 7f1f46000640 1 -- 172.21.15.131:0/1246050266 shutdown_connections 2024-04-28T12:37:28.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.076+0000 7f1f46000640 1 -- 172.21.15.131:0/1246050266 wait complete. 2024-04-28T12:37:28.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.078+0000 7f1f46000640 1 Processor -- start 2024-04-28T12:37:28.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.078+0000 7f1f46000640 1 -- start start 2024-04-28T12:37:28.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f46000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f28004d40 0x7f1f4015c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f46000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400ae560 0x7f1f400ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f46000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f40160760 0x7f1f40160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f46000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4015e0d0 con 0x7f1f28004d40 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f46000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1f40073da0 con 0x7f1f40160760 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f46000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1f40072620 con 0x7f1f400ae560 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f44c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f28004d40 0x7f1f4015c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f44c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f28004d40 0x7f1f4015c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55274/0 (socket says 172.21.15.131:55274) 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f44c00640 1 -- 172.21.15.131:0/2482670711 learned_addr learned my addr 172.21.15.131:0/2482670711 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f45600640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f40160760 0x7f1f40160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f3fe00640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400ae560 0x7f1f400ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f44c00640 1 -- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400ae560 msgr2=0x7f1f400ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f44c00640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400ae560 0x7f1f400ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f44c00640 1 -- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f40160760 msgr2=0x7f1f40160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f44c00640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f40160760 0x7f1f40160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f44c00640 1 -- 172.21.15.131:0/2482670711 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f400af0d0 con 0x7f1f28004d40 2024-04-28T12:37:28.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f3fe00640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400ae560 0x7f1f400ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:28.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.079+0000 7f1f45600640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f40160760 0x7f1f40160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:28.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.080+0000 7f1f44c00640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f28004d40 0x7f1f4015c980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1f34011600 tx=0x7f1f34011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:28.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.080+0000 7f1f3cc00640 1 -- 172.21.15.131:0/2482670711 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400f920 con 0x7f1f28004d40 2024-04-28T12:37:28.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.080+0000 7f1f3cc00640 1 -- 172.21.15.131:0/2482670711 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f3400fac0 con 0x7f1f28004d40 2024-04-28T12:37:28.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.080+0000 7f1f46000640 1 -- 172.21.15.131:0/2482670711 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f400af3b0 con 0x7f1f28004d40 2024-04-28T12:37:28.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.080+0000 7f1f3cc00640 1 -- 172.21.15.131:0/2482670711 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400fe70 con 0x7f1f28004d40 2024-04-28T12:37:28.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.080+0000 7f1f46000640 1 -- 172.21.15.131:0/2482670711 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f40146220 con 0x7f1f28004d40 2024-04-28T12:37:28.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.081+0000 7f1f46000640 1 -- 172.21.15.131:0/2482670711 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f40069350 con 0x7f1f28004d40 2024-04-28T12:37:28.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.082+0000 7f1f3cc00640 1 -- 172.21.15.131:0/2482670711 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f34015460 con 0x7f1f28004d40 2024-04-28T12:37:28.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.083+0000 7f1f3cc00640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1f2006c8f0 0x7f1f2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.083+0000 7f1f3cc00640 1 -- 172.21.15.131:0/2482670711 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1f3408c860 con 0x7f1f28004d40 2024-04-28T12:37:28.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.084+0000 7f1f3fe00640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1f2006c8f0 0x7f1f2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.085+0000 7f1f3fe00640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1f2006c8f0 0x7f1f2006edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0017e0 tx=0x7f1f2c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:28.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.087+0000 7f1f3cc00640 1 -- 172.21.15.131:0/2482670711 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f34019030 con 0x7f1f28004d40 2024-04-28T12:37:28.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.326+0000 7f1f46000640 1 -- 172.21.15.131:0/2482670711 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f400d0670 con 0x7f1f28004d40 2024-04-28T12:37:28.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.327+0000 7f1f3cc00640 1 -- 172.21.15.131:0/2482670711 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5191 (secure 0 0 0) 0x7f1f3405a950 con 0x7f1f28004d40 2024-04-28T12:37:28.324 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:28.324 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1064054651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1064054651},{"type":"v1","addr":"172.21.15.131:6834","nonce":1064054651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/398655652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":398655652},{"type":"v1","addr":"172.21.15.131:6835","nonce":398655652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2619661531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2619661531},{"type":"v1","addr":"172.21.15.157:6837","nonce":2619661531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:37:22.236351+0000","modified":"2024-04-28T12:37:27.318080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1937780695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1937780695},{"type":"v1","addr":"172.21.15.157:6835","nonce":1937780695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34160,"qdb_cluster":[34160]},"id":1}]} 2024-04-28T12:37:28.324 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-04-28T12:37:28.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.330+0000 7f1f46000640 1 -- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1f2006c8f0 msgr2=0x7f1f2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.330+0000 7f1f46000640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1f2006c8f0 0x7f1f2006edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0017e0 tx=0x7f1f2c009040 comp rx=0 tx=0).stop 2024-04-28T12:37:28.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.330+0000 7f1f46000640 1 -- 172.21.15.131:0/2482670711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f28004d40 msgr2=0x7f1f4015c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.330+0000 7f1f46000640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f28004d40 0x7f1f4015c980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1f34011600 tx=0x7f1f34011ac0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.330+0000 7f1f46000640 1 -- 172.21.15.131:0/2482670711 shutdown_connections 2024-04-28T12:37:28.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.330+0000 7f1f46000640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1f2006c8f0 0x7f1f2006edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.330+0000 7f1f46000640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f40160760 0x7f1f40160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.330+0000 7f1f46000640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f400ae560 0x7f1f400ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.330+0000 7f1f46000640 1 --2- 172.21.15.131:0/2482670711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f28004d40 0x7f1f4015c980 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.330+0000 7f1f46000640 1 -- 172.21.15.131:0/2482670711 >> 172.21.15.131:0/2482670711 conn(0x7f1f40068090 msgr2=0x7f1f4015e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:28.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.330+0000 7f1f46000640 1 -- 172.21.15.131:0/2482670711 shutdown_connections 2024-04-28T12:37:28.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.330+0000 7f1f46000640 1 -- 172.21.15.131:0/2482670711 wait complete. 2024-04-28T12:37:28.346 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:28.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.547+0000 7f0e22c00640 1 Processor -- start 2024-04-28T12:37:28.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.547+0000 7f0e22c00640 1 -- start start 2024-04-28T12:37:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.549+0000 7f0e22c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c15f000 0x7f0e1c0d3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.549+0000 7f0e22c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c0b42c0 con 0x7f0e1c0d0810 2024-04-28T12:37:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.549+0000 7f0e22c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c0b3ea0 con 0x7f0e1c0d1d30 2024-04-28T12:37:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.549+0000 7f0e22c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c0b3bc0 con 0x7f0e1c15f000 2024-04-28T12:37:28.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.549+0000 7f0e22200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0e1c0d0810 0x7f0e1c0b4ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54916/0 (socket says 172.21.15.131:54916) 2024-04-28T12:37:28.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.549+0000 7f0e22200640 1 -- 172.21.15.131:0/1970664140 learned_addr learned my addr 172.21.15.131:0/1970664140 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:28.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.549+0000 7f0e21800640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c15f000 0x7f0e1c0d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.549+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2188025939 0 0) 0x7f0e14001490 con 0x7f0e1c0d0810 2024-04-28T12:37:28.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e04003520 con 0x7f0e1c0d0810 2024-04-28T12:37:28.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2823236461 0 0) 0x7f0e04003520 con 0x7f0e1c0d0810 2024-04-28T12:37:28.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c15f000 msgr2=0x7f0e1c0d3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:28.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c15f000 0x7f0e1c0d3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 >> v1:172.21.15.157:6789/0 conn(0x7f0e1c0d1d30 legacy=0x7f0e1c0b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0d57b0 con 0x7f0e1c0d0810 2024-04-28T12:37:28.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f0e14001bc0 con 0x7f0e1c0d0810 2024-04-28T12:37:28.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 >> v1:172.21.15.131:6789/0 conn(0x7f0e1c0d0810 legacy=0x7f0e1c0b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e040039e0 0x7f0e04003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e04004800 0x7f0e04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c15f000 0x7f0e040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c0b3bc0 con 0x7f0e04004800 2024-04-28T12:37:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0400ba70 con 0x7f0e040039e0 2024-04-28T12:37:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0400bc70 con 0x7f0e1c15f000 2024-04-28T12:37:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.550+0000 7f0e20e00640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e04004800 0x7f0e04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.551+0000 7f0e21800640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e040039e0 0x7f0e04003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.551+0000 7f0e22200640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c15f000 0x7f0e040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.551+0000 7f0e20e00640 1 -- 172.21.15.131:0/1970664140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c15f000 msgr2=0x7f0e040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.551+0000 7f0e20e00640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c15f000 0x7f0e040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.551+0000 7f0e20e00640 1 -- 172.21.15.131:0/1970664140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e040039e0 msgr2=0x7f0e04003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.551+0000 7f0e20e00640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e040039e0 0x7f0e04003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.551+0000 7f0e20e00640 1 -- 172.21.15.131:0/1970664140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e04003520 con 0x7f0e04004800 2024-04-28T12:37:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.551+0000 7f0e21800640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e040039e0 0x7f0e04003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:37:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.552+0000 7f0e20e00640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e04004800 0x7f0e04004c80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0d2170 tx=0x7f0e04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867cc074368d5306 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.552+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08001370 con 0x7f0e04004800 2024-04-28T12:37:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.552+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e080052d0 con 0x7f0e04004800 2024-04-28T12:37:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.552+0000 7f0e1be00640 1 -- 172.21.15.131:0/1970664140 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0802ab70 con 0x7f0e04004800 2024-04-28T12:37:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.553+0000 7f0e22c00640 1 -- 172.21.15.131:0/1970664140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e04004800 msgr2=0x7f0e04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.553+0000 7f0e22c00640 1 --2- 172.21.15.131:0/1970664140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e04004800 0x7f0e04004c80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0d2170 tx=0x7f0e04005830 comp rx=0 tx=0).stop 2024-04-28T12:37:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.553+0000 7f0e22200640 1 -- 172.21.15.131:0/1970664140 reap_dead start 2024-04-28T12:37:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.553+0000 7f0e22c00640 1 -- 172.21.15.131:0/1970664140 shutdown_connections 2024-04-28T12:37:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.553+0000 7f0e22c00640 1 -- 172.21.15.131:0/1970664140 >> 172.21.15.131:0/1970664140 conn(0x7f0e1c0d4bf0 msgr2=0x7f0e1c0d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.553+0000 7f0e22c00640 1 -- 172.21.15.131:0/1970664140 shutdown_connections 2024-04-28T12:37:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.553+0000 7f0e22c00640 1 -- 172.21.15.131:0/1970664140 wait complete. 2024-04-28T12:37:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.555+0000 7f0e22c00640 1 Processor -- start 2024-04-28T12:37:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.555+0000 7f0e22c00640 1 -- start start 2024-04-28T12:37:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.556+0000 7f0e22c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e04004800 0x7f0e1c1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.556+0000 7f0e22c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c0b2310 0x7f0e1c0b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.556+0000 7f0e22c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e1c1f5150 0x7f0e1c1f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.556+0000 7f0e22c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c0d2d50 con 0x7f0e1c1f5150 2024-04-28T12:37:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.556+0000 7f0e22c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c0d2a70 con 0x7f0e04004800 2024-04-28T12:37:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.556+0000 7f0e22c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c15f3e0 con 0x7f0e1c0b2310 2024-04-28T12:37:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.556+0000 7f0e22200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e1c1f5150 0x7f0e1c1f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.556+0000 7f0e22200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e1c1f5150 0x7f0e1c1f55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55286/0 (socket says 172.21.15.131:55286) 2024-04-28T12:37:28.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.556+0000 7f0e22200640 1 -- 172.21.15.131:0/1856335466 learned_addr learned my addr 172.21.15.131:0/1856335466 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:28.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e22200640 1 -- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c0b2310 msgr2=0x7f0e1c0b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e21800640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e04004800 0x7f0e1c1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e20e00640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c0b2310 0x7f0e1c0b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e22200640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c0b2310 0x7f0e1c0b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e22200640 1 -- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e04004800 msgr2=0x7f0e1c1571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e22200640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e04004800 0x7f0e1c1571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e22200640 1 -- 172.21.15.131:0/1856335466 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0b8990 con 0x7f0e1c1f5150 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e20e00640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c0b2310 0x7f0e1c0b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e22200640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e1c1f5150 0x7f0e1c1f55d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c08e090 tx=0x7f0e1400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e19600640 1 -- 172.21.15.131:0/1856335466 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14017020 con 0x7f0e1c1f5150 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e19600640 1 -- 172.21.15.131:0/1856335466 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e1400eb90 con 0x7f0e1c1f5150 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e19600640 1 -- 172.21.15.131:0/1856335466 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14013420 con 0x7f0e1c1f5150 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.557+0000 7f0e22c00640 1 -- 172.21.15.131:0/1856335466 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c0b8c70 con 0x7f0e1c1f5150 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.558+0000 7f0e22c00640 1 -- 172.21.15.131:0/1856335466 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c0b9750 con 0x7f0e1c1f5150 2024-04-28T12:37:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.559+0000 7f0e22c00640 1 -- 172.21.15.131:0/1856335466 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e1c069350 con 0x7f0e1c1f5150 2024-04-28T12:37:28.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.559+0000 7f0e19600640 1 -- 172.21.15.131:0/1856335466 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e1401a020 con 0x7f0e1c1f5150 2024-04-28T12:37:28.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.560+0000 7f0e19600640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0dfc06c920 0x7f0dfc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.560+0000 7f0e19600640 1 -- 172.21.15.131:0/1856335466 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0e1408bf80 con 0x7f0e1c1f5150 2024-04-28T12:37:28.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.561+0000 7f0e21800640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0dfc06c920 0x7f0dfc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.562+0000 7f0e21800640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0dfc06c920 0x7f0dfc06ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0e100099b0 tx=0x7f0e10008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:28.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.566+0000 7f0e19600640 1 -- 172.21.15.131:0/1856335466 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e14021030 con 0x7f0e1c1f5150 2024-04-28T12:37:28.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.765+0000 7f0e22c00640 1 -- 172.21.15.131:0/1856335466 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e1c15e800 con 0x7f0e1c1f5150 2024-04-28T12:37:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.767+0000 7f0e19600640 1 -- 172.21.15.131:0/1856335466 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7f0e1c0b42c0 con 0x7f0e1c1f5150 2024-04-28T12:37:28.765 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:28.765 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:28.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.770+0000 7f0e22c00640 1 -- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0dfc06c920 msgr2=0x7f0dfc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.770+0000 7f0e22c00640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0dfc06c920 0x7f0dfc06ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0e100099b0 tx=0x7f0e10008040 comp rx=0 tx=0).stop 2024-04-28T12:37:28.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.770+0000 7f0e22c00640 1 -- 172.21.15.131:0/1856335466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e1c1f5150 msgr2=0x7f0e1c1f55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.770+0000 7f0e22c00640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e1c1f5150 0x7f0e1c1f55d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c08e090 tx=0x7f0e1400e530 comp rx=0 tx=0).stop 2024-04-28T12:37:28.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.770+0000 7f0e22c00640 1 -- 172.21.15.131:0/1856335466 shutdown_connections 2024-04-28T12:37:28.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.770+0000 7f0e22c00640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0dfc06c920 0x7f0dfc06ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.770+0000 7f0e22c00640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e1c1f5150 0x7f0e1c1f55d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.770+0000 7f0e22c00640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1c0b2310 0x7f0e1c0b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.770+0000 7f0e22c00640 1 --2- 172.21.15.131:0/1856335466 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e04004800 0x7f0e1c1571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.770+0000 7f0e22c00640 1 -- 172.21.15.131:0/1856335466 >> 172.21.15.131:0/1856335466 conn(0x7f0e1c0d4bf0 msgr2=0x7f0e1c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.770+0000 7f0e22c00640 1 -- 172.21.15.131:0/1856335466 shutdown_connections 2024-04-28T12:37:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.770+0000 7f0e22c00640 1 -- 172.21.15.131:0/1856335466 wait complete. 2024-04-28T12:37:28.784 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:28.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.975+0000 7fd9f9a00640 1 Processor -- start 2024-04-28T12:37:28.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.975+0000 7fd9f9a00640 1 -- start start 2024-04-28T12:37:28.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.977+0000 7fd9f9a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9f41467c0 0x7fd9f4072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.977+0000 7fd9f9a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f40b4140 0x7fd9f406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.977+0000 7fd9f9a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f4146ba0 con 0x7fd9f40b3ce0 2024-04-28T12:37:28.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.977+0000 7fd9f9a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f4073dd0 con 0x7fd9f41467c0 2024-04-28T12:37:28.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.977+0000 7fd9f9a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f40731c0 con 0x7fd9f40b4140 2024-04-28T12:37:28.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.977+0000 7fd9f2000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd9f40b3ce0 0x7fd9f40d0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54922/0 (socket says 172.21.15.131:54922) 2024-04-28T12:37:28.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.977+0000 7fd9f2000640 1 -- 172.21.15.131:0/1867794658 learned_addr learned my addr 172.21.15.131:0/1867794658 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:28.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.977+0000 7fd9f2a00640 1 --2- 172.21.15.131:0/1867794658 >> v2:172.21.15.157:3300/0 conn(0x7fd9f41467c0 0x7fd9f4072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.977+0000 7fd9f3400640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f40b4140 0x7fd9f406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1230202215 0 0) 0x7fd9f4146ba0 con 0x7fd9f40b3ce0 2024-04-28T12:37:28.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9d8003520 con 0x7fd9f40b3ce0 2024-04-28T12:37:28.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1550664043 0 0) 0x7fd9d8003520 con 0x7fd9f40b3ce0 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f40b4140 msgr2=0x7fd9f406dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f40b4140 0x7fd9f406dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 >> v2:172.21.15.157:3300/0 conn(0x7fd9f41467c0 msgr2=0x7fd9f4072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 --2- 172.21.15.131:0/1867794658 >> v2:172.21.15.157:3300/0 conn(0x7fd9f41467c0 0x7fd9f4072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f40734a0 con 0x7fd9f40b3ce0 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fd9e0001670 con 0x7fd9f40b3ce0 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 >> v1:172.21.15.131:6789/0 conn(0x7fd9f40b3ce0 legacy=0x7fd9f40d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9d80039b0 0x7fd9d8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d8004800 0x7fd9d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8005180 0x7fd9d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4073dd0 con 0x7fd9d80039b0 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f40731c0 con 0x7fd9d8005180 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d800bf00 con 0x7fd9d8004800 2024-04-28T12:37:28.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.978+0000 7fd9f2a00640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9d80039b0 0x7fd9d8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.979+0000 7fd9f3400640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8005180 0x7fd9d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.979+0000 7fd9f2000640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d8004800 0x7fd9d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.979+0000 7fd9f2a00640 1 -- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d8004800 msgr2=0x7fd9d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:28.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.979+0000 7fd9f2a00640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d8004800 0x7fd9d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.979+0000 7fd9f2a00640 1 -- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8005180 msgr2=0x7fd9d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:28.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.979+0000 7fd9f2a00640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8005180 0x7fd9d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.979+0000 7fd9f2a00640 1 -- 172.21.15.131:0/1867794658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d8003520 con 0x7fd9d80039b0 2024-04-28T12:37:28.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.979+0000 7fd9f3400640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9d8005180 0x7fd9d800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:28.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.979+0000 7fd9f2000640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9d8004800 0x7fd9d8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:28.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.980+0000 7fd9f2a00640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9d80039b0 0x7fd9d8003df0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009ac0 tx=0x7fd9d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944303f7a6692faa server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.980+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4036070 con 0x7fd9d80039b0 2024-04-28T12:37:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.980+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e4031400 con 0x7fd9d80039b0 2024-04-28T12:37:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.980+0000 7fd9f1600640 1 -- 172.21.15.131:0/1867794658 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4009690 con 0x7fd9d80039b0 2024-04-28T12:37:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.981+0000 7fd9f9a00640 1 -- 172.21.15.131:0/1867794658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9d80039b0 msgr2=0x7fd9d8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.981+0000 7fd9f9a00640 1 --2- 172.21.15.131:0/1867794658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9d80039b0 0x7fd9d8003df0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009ac0 tx=0x7fd9d8005830 comp rx=0 tx=0).stop 2024-04-28T12:37:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.981+0000 7fd9f3400640 1 -- 172.21.15.131:0/1867794658 reap_dead start 2024-04-28T12:37:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.981+0000 7fd9f9a00640 1 -- 172.21.15.131:0/1867794658 shutdown_connections 2024-04-28T12:37:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.981+0000 7fd9f9a00640 1 -- 172.21.15.131:0/1867794658 >> 172.21.15.131:0/1867794658 conn(0x7fd9f40b2900 msgr2=0x7fd9f40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.981+0000 7fd9f9a00640 1 -- 172.21.15.131:0/1867794658 shutdown_connections 2024-04-28T12:37:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.981+0000 7fd9f9a00640 1 -- 172.21.15.131:0/1867794658 wait complete. 2024-04-28T12:37:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.983+0000 7fd9f9a00640 1 Processor -- start 2024-04-28T12:37:28.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.983+0000 7fd9f9a00640 1 -- start start 2024-04-28T12:37:28.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f9a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9d80039b0 0x7fd9f4142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f9a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f40c5ce0 0x7fd9f40c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f9a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f40b1040 0x7fd9f40b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f9a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f40d1230 con 0x7fd9d80039b0 2024-04-28T12:37:28.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f9a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4073a60 con 0x7fd9f40b1040 2024-04-28T12:37:28.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f9a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f40d13b0 con 0x7fd9f40c5ce0 2024-04-28T12:37:28.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9d80039b0 0x7fd9f4142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9d80039b0 0x7fd9f4142080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55304/0 (socket says 172.21.15.131:55304) 2024-04-28T12:37:28.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f2a00640 1 -- 172.21.15.131:0/3809157943 learned_addr learned my addr 172.21.15.131:0/3809157943 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:28.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f2000640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f40c5ce0 0x7fd9f40c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f2a00640 1 -- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f40c5ce0 msgr2=0x7fd9f40c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f3400640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f40b1040 0x7fd9f40b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f2a00640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f40c5ce0 0x7fd9f40c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f2a00640 1 -- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f40b1040 msgr2=0x7fd9f40b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:28.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f2a00640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f40b1040 0x7fd9f40b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:28.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.984+0000 7fd9f2a00640 1 -- 172.21.15.131:0/3809157943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f41f7e20 con 0x7fd9d80039b0 2024-04-28T12:37:28.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.985+0000 7fd9f2a00640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9d80039b0 0x7fd9f4142080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009ac0 tx=0x7fd9e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:28.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.985+0000 7fd9eaa00640 1 -- 172.21.15.131:0/3809157943 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4040020 con 0x7fd9d80039b0 2024-04-28T12:37:28.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.985+0000 7fd9eaa00640 1 -- 172.21.15.131:0/3809157943 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e4003b40 con 0x7fd9d80039b0 2024-04-28T12:37:28.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.985+0000 7fd9f9a00640 1 -- 172.21.15.131:0/3809157943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f41f8100 con 0x7fd9d80039b0 2024-04-28T12:37:28.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.985+0000 7fd9eaa00640 1 -- 172.21.15.131:0/3809157943 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4036040 con 0x7fd9d80039b0 2024-04-28T12:37:28.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.985+0000 7fd9f9a00640 1 -- 172.21.15.131:0/3809157943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f41f86c0 con 0x7fd9d80039b0 2024-04-28T12:37:28.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.986+0000 7fd9f9a00640 1 -- 172.21.15.131:0/3809157943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f4069350 con 0x7fd9d80039b0 2024-04-28T12:37:28.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.987+0000 7fd9eaa00640 1 -- 172.21.15.131:0/3809157943 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9e4043020 con 0x7fd9d80039b0 2024-04-28T12:37:28.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.988+0000 7fd9eaa00640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd9d406c920 0x7fd9d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:28.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.988+0000 7fd9eaa00640 1 -- 172.21.15.131:0/3809157943 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd9e40abff0 con 0x7fd9d80039b0 2024-04-28T12:37:28.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.989+0000 7fd9f2000640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd9d406c920 0x7fd9d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:28.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.990+0000 7fd9f2000640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd9d406c920 0x7fd9d406ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0001eb0 tx=0x7fd9e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:28.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:28.993+0000 7fd9eaa00640 1 -- 172.21.15.131:0/3809157943 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd9e404a070 con 0x7fd9d80039b0 2024-04-28T12:37:29.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.192+0000 7fd9f9a00640 1 -- 172.21.15.131:0/3809157943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9f40d0bc0 con 0x7fd9d80039b0 2024-04-28T12:37:29.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.196+0000 7fd9eaa00640 1 -- 172.21.15.131:0/3809157943 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7fd9e40044e0 con 0x7fd9d80039b0 2024-04-28T12:37:29.193 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:29.193 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:29.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.198+0000 7fd9f9a00640 1 -- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd9d406c920 msgr2=0x7fd9d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.198+0000 7fd9f9a00640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd9d406c920 0x7fd9d406ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0001eb0 tx=0x7fd9e0009040 comp rx=0 tx=0).stop 2024-04-28T12:37:29.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.198+0000 7fd9f9a00640 1 -- 172.21.15.131:0/3809157943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9d80039b0 msgr2=0x7fd9f4142080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.198+0000 7fd9f9a00640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9d80039b0 0x7fd9f4142080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009ac0 tx=0x7fd9e4004730 comp rx=0 tx=0).stop 2024-04-28T12:37:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.198+0000 7fd9f9a00640 1 -- 172.21.15.131:0/3809157943 shutdown_connections 2024-04-28T12:37:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.198+0000 7fd9f9a00640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd9d406c920 0x7fd9d406ede0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.198+0000 7fd9f9a00640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9f40b1040 0x7fd9f40b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.199+0000 7fd9f9a00640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9f40c5ce0 0x7fd9f40c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.199+0000 7fd9f9a00640 1 --2- 172.21.15.131:0/3809157943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9d80039b0 0x7fd9f4142080 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.199+0000 7fd9f9a00640 1 -- 172.21.15.131:0/3809157943 >> 172.21.15.131:0/3809157943 conn(0x7fd9f40b2900 msgr2=0x7fd9f4145af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.199+0000 7fd9f9a00640 1 -- 172.21.15.131:0/3809157943 shutdown_connections 2024-04-28T12:37:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.199+0000 7fd9f9a00640 1 -- 172.21.15.131:0/3809157943 wait complete. 2024-04-28T12:37:29.231 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:37:29.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.435+0000 7f1c7f400640 1 Processor -- start 2024-04-28T12:37:29.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.435+0000 7f1c7f400640 1 -- start start 2024-04-28T12:37:29.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.437+0000 7f1c7f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7815cab0 0x7f1c780afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.437+0000 7f1c7f400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c780b0020 0x7f1c780d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.437+0000 7f1c7f400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c780b0650 con 0x7f1c780d1080 2024-04-28T12:37:29.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.437+0000 7f1c7f400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1c780d0900 con 0x7f1c780b0020 2024-04-28T12:37:29.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.437+0000 7f1c7f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1c781635a0 con 0x7f1c7815cab0 2024-04-28T12:37:29.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.438+0000 7f1c77e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c780b0020 0x7f1c780d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.438+0000 7f1c7ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7815cab0 0x7f1c780afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.438+0000 7f1c77e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1c780b0020 0x7f1c780d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:50034/0 (socket says 172.21.15.131:50034) 2024-04-28T12:37:29.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.438+0000 7f1c77e00640 1 -- 172.21.15.131:0/299910156 learned_addr learned my addr 172.21.15.131:0/299910156 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:29.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.438+0000 7f1c77e00640 1 -- 172.21.15.131:0/299910156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7815cab0 msgr2=0x7f1c780afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:29.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.438+0000 7f1c77e00640 1 --2- 172.21.15.131:0/299910156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7815cab0 0x7f1c780afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.438+0000 7f1c77e00640 1 -- 172.21.15.131:0/299910156 >> v1:172.21.15.157:6789/0 conn(0x7f1c780d1080 legacy=0x7f1c780ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.439+0000 7f1c77e00640 1 -- 172.21.15.131:0/299910156 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c78163a30 con 0x7f1c780b0020 2024-04-28T12:37:29.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.440+0000 7f1c77e00640 1 --2- 172.21.15.131:0/299910156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c780b0020 0x7f1c780d0b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009870 tx=0x7f1c6402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f4fcc5e02a6e798 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:29.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.440+0000 7f1c77400640 1 -- 172.21.15.131:0/299910156 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64037070 con 0x7f1c780b0020 2024-04-28T12:37:29.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.440+0000 7f1c77400640 1 -- 172.21.15.131:0/299910156 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c64032440 con 0x7f1c780b0020 2024-04-28T12:37:29.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.440+0000 7f1c77400640 1 -- 172.21.15.131:0/299910156 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64032810 con 0x7f1c780b0020 2024-04-28T12:37:29.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.441+0000 7f1c7f400640 1 -- 172.21.15.131:0/299910156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c780b0020 msgr2=0x7f1c780d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.441+0000 7f1c7f400640 1 --2- 172.21.15.131:0/299910156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c780b0020 0x7f1c780d0b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009870 tx=0x7f1c6402a270 comp rx=0 tx=0).stop 2024-04-28T12:37:29.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.441+0000 7f1c7f400640 1 -- 172.21.15.131:0/299910156 shutdown_connections 2024-04-28T12:37:29.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.441+0000 7f1c7f400640 1 --2- 172.21.15.131:0/299910156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c780b0020 0x7f1c780d0b40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.441+0000 7f1c7f400640 1 --2- 172.21.15.131:0/299910156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c7815cab0 0x7f1c780afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.441+0000 7f1c7f400640 1 -- 172.21.15.131:0/299910156 >> 172.21.15.131:0/299910156 conn(0x7f1c780b2900 msgr2=0x7f1c780b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:29.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.441+0000 7f1c7f400640 1 -- 172.21.15.131:0/299910156 shutdown_connections 2024-04-28T12:37:29.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.441+0000 7f1c7f400640 1 -- 172.21.15.131:0/299910156 wait complete. 2024-04-28T12:37:29.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.443+0000 7f1c7f400640 1 Processor -- start 2024-04-28T12:37:29.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.444+0000 7f1c7f400640 1 -- start start 2024-04-28T12:37:29.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.445+0000 7f1c7f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c780b0020 0x7f1c7806fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.445+0000 7f1c7f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c780d1080 0x7f1c78155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.445+0000 7f1c7f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7815cab0 0x7f1c78155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.445+0000 7f1c7f400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1c78163230 con 0x7f1c7815cab0 2024-04-28T12:37:29.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.445+0000 7f1c7f400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1c78162f50 con 0x7f1c780b0020 2024-04-28T12:37:29.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.445+0000 7f1c7f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1c7815ce90 con 0x7f1c780d1080 2024-04-28T12:37:29.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.445+0000 7f1c7d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7815cab0 0x7f1c78155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.445+0000 7f1c7d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7815cab0 0x7f1c78155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55306/0 (socket says 172.21.15.131:55306) 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.445+0000 7f1c7d400640 1 -- 172.21.15.131:0/1545893697 learned_addr learned my addr 172.21.15.131:0/1545893697 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.445+0000 7f1c7ca00640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c780b0020 0x7f1c7806fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.445+0000 7f1c77e00640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c780d1080 0x7f1c78155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.446+0000 7f1c7d400640 1 -- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c780d1080 msgr2=0x7f1c78155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.446+0000 7f1c7d400640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c780d1080 0x7f1c78155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.446+0000 7f1c7d400640 1 -- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c780b0020 msgr2=0x7f1c7806fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.446+0000 7f1c7d400640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c780b0020 0x7f1c7806fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.446+0000 7f1c7d400640 1 -- 172.21.15.131:0/1545893697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7806e2f0 con 0x7f1c7815cab0 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.446+0000 7f1c7d400640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7815cab0 0x7f1c78155b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000c7d0 tx=0x7f1c7000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.446+0000 7f1c74c00640 1 -- 172.21.15.131:0/1545893697 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70003b70 con 0x7f1c7815cab0 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.446+0000 7f1c74c00640 1 -- 172.21.15.131:0/1545893697 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c70003d30 con 0x7f1c7815cab0 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.446+0000 7f1c7f400640 1 -- 172.21.15.131:0/1545893697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7806e5d0 con 0x7f1c7815cab0 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.446+0000 7f1c74c00640 1 -- 172.21.15.131:0/1545893697 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7000aa90 con 0x7f1c7815cab0 2024-04-28T12:37:29.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.446+0000 7f1c7f400640 1 -- 172.21.15.131:0/1545893697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7806eb90 con 0x7f1c7815cab0 2024-04-28T12:37:29.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.447+0000 7f1c7f400640 1 -- 172.21.15.131:0/1545893697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c78069350 con 0x7f1c7815cab0 2024-04-28T12:37:29.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.448+0000 7f1c74c00640 1 -- 172.21.15.131:0/1545893697 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c7000ac30 con 0x7f1c7815cab0 2024-04-28T12:37:29.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.449+0000 7f1c74c00640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1c5806c920 0x7f1c5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.449+0000 7f1c74c00640 1 -- 172.21.15.131:0/1545893697 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1c7008bd70 con 0x7f1c7815cab0 2024-04-28T12:37:29.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.450+0000 7f1c7ca00640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1c5806c920 0x7f1c5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.451+0000 7f1c7ca00640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1c5806c920 0x7f1c5806ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009aa0 tx=0x7f1c6c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:29.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.454+0000 7f1c74c00640 1 -- 172.21.15.131:0/1545893697 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c700596f0 con 0x7f1c7815cab0 2024-04-28T12:37:29.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.690+0000 7f1c7f400640 1 -- 172.21.15.131:0/1545893697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c7806faf0 con 0x7f1c7815cab0 2024-04-28T12:37:29.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.691+0000 7f1c74c00640 1 -- 172.21.15.131:0/1545893697 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5191 (secure 0 0 0) 0x7f1c7005d9f0 con 0x7f1c7815cab0 2024-04-28T12:37:29.688 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:29.688 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1064054651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1064054651},{"type":"v1","addr":"172.21.15.131:6834","nonce":1064054651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/398655652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":398655652},{"type":"v1","addr":"172.21.15.131:6835","nonce":398655652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2619661531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2619661531},{"type":"v1","addr":"172.21.15.157:6837","nonce":2619661531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:37:22.236351+0000","modified":"2024-04-28T12:37:27.318080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1937780695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1937780695},{"type":"v1","addr":"172.21.15.157:6835","nonce":1937780695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34160,"qdb_cluster":[34160]},"id":1}]} 2024-04-28T12:37:29.688 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-04-28T12:37:29.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.694+0000 7f1c7f400640 1 -- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1c5806c920 msgr2=0x7f1c5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.694+0000 7f1c7f400640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1c5806c920 0x7f1c5806ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009aa0 tx=0x7f1c6c009190 comp rx=0 tx=0).stop 2024-04-28T12:37:29.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.694+0000 7f1c7f400640 1 -- 172.21.15.131:0/1545893697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7815cab0 msgr2=0x7f1c78155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.694+0000 7f1c7f400640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7815cab0 0x7f1c78155b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000c7d0 tx=0x7f1c7000cc90 comp rx=0 tx=0).stop 2024-04-28T12:37:29.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.694+0000 7f1c7f400640 1 -- 172.21.15.131:0/1545893697 shutdown_connections 2024-04-28T12:37:29.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.694+0000 7f1c7f400640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1c5806c920 0x7f1c5806ede0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.694+0000 7f1c7f400640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c7815cab0 0x7f1c78155b70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.694+0000 7f1c7f400640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c780d1080 0x7f1c78155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.694+0000 7f1c7f400640 1 --2- 172.21.15.131:0/1545893697 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1c780b0020 0x7f1c7806fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.694+0000 7f1c7f400640 1 -- 172.21.15.131:0/1545893697 >> 172.21.15.131:0/1545893697 conn(0x7f1c780b2900 msgr2=0x7f1c7815d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:29.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.694+0000 7f1c7f400640 1 -- 172.21.15.131:0/1545893697 shutdown_connections 2024-04-28T12:37:29.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.694+0000 7f1c7f400640 1 -- 172.21.15.131:0/1545893697 wait complete. 2024-04-28T12:37:29.710 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:29.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.911+0000 7ff31c200640 1 Processor -- start 2024-04-28T12:37:29.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.911+0000 7ff31c200640 1 -- start start 2024-04-28T12:37:29.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.912+0000 7ff31c200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3140b3490 0x7ff31406f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.912+0000 7ff31c200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3140cf060 con 0x7ff314145590 2024-04-28T12:37:29.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.913+0000 7ff31c200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3140cf340 con 0x7ff31406f560 2024-04-28T12:37:29.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.913+0000 7ff31c200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff3140d0100 con 0x7ff3140b3490 2024-04-28T12:37:29.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.913+0000 7ff319800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff314145590 0x7ff3140ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54926/0 (socket says 172.21.15.131:54926) 2024-04-28T12:37:29.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.913+0000 7ff319800640 1 -- 172.21.15.131:0/1195069846 learned_addr learned my addr 172.21.15.131:0/1195069846 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:29.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.913+0000 7ff318e00640 1 --2- 172.21.15.131:0/1195069846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3140b3490 0x7ff31406f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.913+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3152233065 0 0) 0x7ff3140cf060 con 0x7ff314145590 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.913+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2fc005730 con 0x7ff314145590 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1516842198 0 0) 0x7ff2fc005730 con 0x7ff314145590 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3140b3490 msgr2=0x7ff31406f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 --2- 172.21.15.131:0/1195069846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3140b3490 0x7ff31406f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 >> v1:172.21.15.157:6789/0 conn(0x7ff31406f560 legacy=0x7ff314071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3140cf620 con 0x7ff314145590 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7ff308001670 con 0x7ff314145590 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 >> v1:172.21.15.131:6789/0 conn(0x7ff314145590 legacy=0x7ff3140ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 --2- 172.21.15.131:0/1195069846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2fc005bf0 0x7ff2fc006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 --2- 172.21.15.131:0/1195069846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3140b3490 0x7ff2fc006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 --2- 172.21.15.131:0/1195069846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2fc006f50 0x7ff2fc0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff3140d0100 con 0x7ff2fc005bf0 2024-04-28T12:37:29.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc00dc80 con 0x7ff2fc006f50 2024-04-28T12:37:29.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff2fc00de80 con 0x7ff3140b3490 2024-04-28T12:37:29.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff319800640 1 --2- 172.21.15.131:0/1195069846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2fc005bf0 0x7ff2fc006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff318e00640 1 --2- 172.21.15.131:0/1195069846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3140b3490 0x7ff2fc006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.914+0000 7ff31a200640 1 --2- 172.21.15.131:0/1195069846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2fc006f50 0x7ff2fc0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.915+0000 7ff319800640 1 -- 172.21.15.131:0/1195069846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3140b3490 msgr2=0x7ff2fc006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:29.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.915+0000 7ff319800640 1 --2- 172.21.15.131:0/1195069846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3140b3490 0x7ff2fc006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.915+0000 7ff319800640 1 -- 172.21.15.131:0/1195069846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2fc006f50 msgr2=0x7ff2fc0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:29.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.915+0000 7ff319800640 1 --2- 172.21.15.131:0/1195069846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2fc006f50 0x7ff2fc0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.915+0000 7ff319800640 1 -- 172.21.15.131:0/1195069846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc005730 con 0x7ff2fc005bf0 2024-04-28T12:37:29.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.916+0000 7ff319800640 1 --2- 172.21.15.131:0/1195069846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2fc005bf0 0x7ff2fc006010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c210 tx=0x7ff2fc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89958bc4cbc24cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:29.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.916+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308007cf0 con 0x7ff2fc005bf0 2024-04-28T12:37:29.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.916+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff308005af0 con 0x7ff2fc005bf0 2024-04-28T12:37:29.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.916+0000 7ff313e00640 1 -- 172.21.15.131:0/1195069846 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308007070 con 0x7ff2fc005bf0 2024-04-28T12:37:29.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.916+0000 7ff31c200640 1 -- 172.21.15.131:0/1195069846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2fc005bf0 msgr2=0x7ff2fc006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.916+0000 7ff31c200640 1 --2- 172.21.15.131:0/1195069846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2fc005bf0 0x7ff2fc006010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c210 tx=0x7ff2fc007a40 comp rx=0 tx=0).stop 2024-04-28T12:37:29.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.916+0000 7ff31a200640 1 -- 172.21.15.131:0/1195069846 reap_dead start 2024-04-28T12:37:29.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.917+0000 7ff31c200640 1 -- 172.21.15.131:0/1195069846 shutdown_connections 2024-04-28T12:37:29.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.917+0000 7ff31c200640 1 -- 172.21.15.131:0/1195069846 >> 172.21.15.131:0/1195069846 conn(0x7ff3140b2900 msgr2=0x7ff3140b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:29.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.917+0000 7ff31c200640 1 -- 172.21.15.131:0/1195069846 shutdown_connections 2024-04-28T12:37:29.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.917+0000 7ff31c200640 1 -- 172.21.15.131:0/1195069846 wait complete. 2024-04-28T12:37:29.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.918+0000 7ff31c200640 1 Processor -- start 2024-04-28T12:37:29.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.919+0000 7ff31c200640 1 -- start start 2024-04-28T12:37:29.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.919+0000 7ff31c200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2fc005bf0 0x7ff3141410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.919+0000 7ff31c200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff314072980 0x7ff314072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff31c200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3141426f0 0x7ff314142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff31c200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff314145970 con 0x7ff2fc005bf0 2024-04-28T12:37:29.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff31c200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff3140cfbe0 con 0x7ff314072980 2024-04-28T12:37:29.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff31c200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff3140725c0 con 0x7ff3141426f0 2024-04-28T12:37:29.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff319800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2fc005bf0 0x7ff3141410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff319800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2fc005bf0 0x7ff3141410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55320/0 (socket says 172.21.15.131:55320) 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff319800640 1 -- 172.21.15.131:0/3218577216 learned_addr learned my addr 172.21.15.131:0/3218577216 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff31a200640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3141426f0 0x7ff314142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff318e00640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff314072980 0x7ff314072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff319800640 1 -- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3141426f0 msgr2=0x7ff314142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff319800640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3141426f0 0x7ff314142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff319800640 1 -- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff314072980 msgr2=0x7ff314072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff319800640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff314072980 0x7ff314072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.920+0000 7ff319800640 1 -- 172.21.15.131:0/3218577216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff314141970 con 0x7ff2fc005bf0 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.921+0000 7ff319800640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2fc005bf0 0x7ff3141410b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff30802add0 tx=0x7ff3140cf7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.921+0000 7ff311600640 1 -- 172.21.15.131:0/3218577216 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308041030 con 0x7ff2fc005bf0 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.921+0000 7ff311600640 1 -- 172.21.15.131:0/3218577216 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff308038040 con 0x7ff2fc005bf0 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.921+0000 7ff31c200640 1 -- 172.21.15.131:0/3218577216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff314141c50 con 0x7ff2fc005bf0 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.921+0000 7ff311600640 1 -- 172.21.15.131:0/3218577216 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308001650 con 0x7ff2fc005bf0 2024-04-28T12:37:29.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.921+0000 7ff31c200640 1 -- 172.21.15.131:0/3218577216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3140c5b70 con 0x7ff2fc005bf0 2024-04-28T12:37:29.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.922+0000 7ff31c200640 1 -- 172.21.15.131:0/3218577216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff314076260 con 0x7ff2fc005bf0 2024-04-28T12:37:29.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.923+0000 7ff311600640 1 -- 172.21.15.131:0/3218577216 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff308044070 con 0x7ff2fc005bf0 2024-04-28T12:37:29.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.924+0000 7ff311600640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff2f806c8f0 0x7ff2f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:29.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.924+0000 7ff311600640 1 -- 172.21.15.131:0/3218577216 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff3080ab870 con 0x7ff2fc005bf0 2024-04-28T12:37:29.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.925+0000 7ff318e00640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff2f806c8f0 0x7ff2f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:29.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.925+0000 7ff318e00640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff2f806c8f0 0x7ff2f806edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff314144f30 tx=0x7ff304008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:29.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:29.928+0000 7ff311600640 1 -- 172.21.15.131:0/3218577216 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff314076260 con 0x7ff2fc005bf0 2024-04-28T12:37:30.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.136+0000 7ff31c200640 1 -- 172.21.15.131:0/3218577216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff314069350 con 0x7ff2fc005bf0 2024-04-28T12:37:30.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.139+0000 7ff311600640 1 -- 172.21.15.131:0/3218577216 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7ff30804b020 con 0x7ff2fc005bf0 2024-04-28T12:37:30.136 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:30.136 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:30.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.142+0000 7ff31c200640 1 -- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff2f806c8f0 msgr2=0x7ff2f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:30.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.142+0000 7ff31c200640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff2f806c8f0 0x7ff2f806edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff314144f30 tx=0x7ff304008040 comp rx=0 tx=0).stop 2024-04-28T12:37:30.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.142+0000 7ff31c200640 1 -- 172.21.15.131:0/3218577216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2fc005bf0 msgr2=0x7ff3141410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:30.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.142+0000 7ff31c200640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2fc005bf0 0x7ff3141410b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff30802add0 tx=0x7ff3140cf7f0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.142+0000 7ff31c200640 1 -- 172.21.15.131:0/3218577216 shutdown_connections 2024-04-28T12:37:30.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.142+0000 7ff31c200640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff2f806c8f0 0x7ff2f806edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.142+0000 7ff31c200640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3141426f0 0x7ff314142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.142+0000 7ff31c200640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff314072980 0x7ff314072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.142+0000 7ff31c200640 1 --2- 172.21.15.131:0/3218577216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2fc005bf0 0x7ff3141410b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.142+0000 7ff31c200640 1 -- 172.21.15.131:0/3218577216 >> 172.21.15.131:0/3218577216 conn(0x7ff3140b2900 msgr2=0x7ff3140afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:30.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.142+0000 7ff31c200640 1 -- 172.21.15.131:0/3218577216 shutdown_connections 2024-04-28T12:37:30.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.142+0000 7ff31c200640 1 -- 172.21.15.131:0/3218577216 wait complete. 2024-04-28T12:37:30.562 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-28T12:37:22.236351+0000', 'modified': '2024-04-28T12:37:27.318080+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34160}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34160': {'gid': 34160, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/1937780695', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 1937780695}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 1937780695}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34160, 'qdb_cluster': [34160]} 2024-04-28T12:37:30.562 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-28T12:37:30.562 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-28T12:37:30.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.782+0000 7f1303600640 1 Processor -- start 2024-04-28T12:37:30.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.782+0000 7f1303600640 1 -- start start 2024-04-28T12:37:30.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.784+0000 7f1303600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f12fc14ceb0 0x7f12fc0b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.784+0000 7f1303600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f12fc0cfe60 0x7f12fc16d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.784+0000 7f1303600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12fc14d290 con 0x7f12fc0d1570 2024-04-28T12:37:30.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.784+0000 7f1303600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f12fc0b5140 con 0x7f12fc0cfe60 2024-04-28T12:37:30.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.784+0000 7f1303600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f12fc0b5420 con 0x7f12fc14ceb0 2024-04-28T12:37:30.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.785+0000 7f1301600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f12fc0cfe60 0x7f12fc16d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.785+0000 7f1301600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f12fc0cfe60 0x7f12fc16d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55326/0 (socket says 172.21.15.131:55326) 2024-04-28T12:37:30.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.785+0000 7f1301600640 1 -- 172.21.15.131:0/2208962426 learned_addr learned my addr 172.21.15.131:0/2208962426 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:30.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.785+0000 7f1300c00640 1 --2- 172.21.15.131:0/2208962426 >> v2:172.21.15.157:3300/0 conn(0x7f12fc14ceb0 0x7f12fc0b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.785+0000 7f1301600640 1 -- 172.21.15.131:0/2208962426 >> v2:172.21.15.157:3300/0 conn(0x7f12fc14ceb0 msgr2=0x7f12fc0b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:30.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.785+0000 7f1301600640 1 --2- 172.21.15.131:0/2208962426 >> v2:172.21.15.157:3300/0 conn(0x7f12fc14ceb0 0x7f12fc0b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.785+0000 7f1301600640 1 -- 172.21.15.131:0/2208962426 >> v1:172.21.15.157:6789/0 conn(0x7f12fc0d1570 legacy=0x7f12fc0cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:30.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.785+0000 7f1301600640 1 -- 172.21.15.131:0/2208962426 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc0b5700 con 0x7f12fc0cfe60 2024-04-28T12:37:30.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.786+0000 7f1301600640 1 --2- 172.21.15.131:0/2208962426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12fc0cfe60 0x7f12fc16d040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f12fc06d120 tx=0x7f12f402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cd7188cc7e54e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:30.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.787+0000 7f12fbe00640 1 -- 172.21.15.131:0/2208962426 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4036070 con 0x7f12fc0cfe60 2024-04-28T12:37:30.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.787+0000 7f12fbe00640 1 -- 172.21.15.131:0/2208962426 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f4031600 con 0x7f12fc0cfe60 2024-04-28T12:37:30.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.787+0000 7f12fbe00640 1 -- 172.21.15.131:0/2208962426 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f40319d0 con 0x7f12fc0cfe60 2024-04-28T12:37:30.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.787+0000 7f1303600640 1 -- 172.21.15.131:0/2208962426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12fc0cfe60 msgr2=0x7f12fc16d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:30.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.787+0000 7f1303600640 1 --2- 172.21.15.131:0/2208962426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12fc0cfe60 0x7f12fc16d040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f12fc06d120 tx=0x7f12f402a6f0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.787+0000 7f1303600640 1 -- 172.21.15.131:0/2208962426 shutdown_connections 2024-04-28T12:37:30.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.787+0000 7f1303600640 1 --2- 172.21.15.131:0/2208962426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12fc0cfe60 0x7f12fc16d040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.787+0000 7f1303600640 1 --2- 172.21.15.131:0/2208962426 >> v2:172.21.15.157:3300/0 conn(0x7f12fc14ceb0 0x7f12fc0b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.787+0000 7f1303600640 1 -- 172.21.15.131:0/2208962426 >> 172.21.15.131:0/2208962426 conn(0x7f12fc0b2900 msgr2=0x7f12fc0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:30.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.787+0000 7f1303600640 1 -- 172.21.15.131:0/2208962426 shutdown_connections 2024-04-28T12:37:30.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.787+0000 7f1303600640 1 -- 172.21.15.131:0/2208962426 wait complete. 2024-04-28T12:37:30.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.789+0000 7f1303600640 1 Processor -- start 2024-04-28T12:37:30.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.789+0000 7f1303600640 1 -- start start 2024-04-28T12:37:30.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.790+0000 7f1303600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc0cfe60 0x7f12fc0acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.790+0000 7f1303600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12fc0d1570 0x7f12fc0aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.790+0000 7f1303600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc14ceb0 0x7f12fc0ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.790+0000 7f1303600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc0d1a00 con 0x7f12fc0d1570 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.790+0000 7f1303600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc0b5cc0 con 0x7f12fc14ceb0 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.790+0000 7f1303600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc06f000 con 0x7f12fc0cfe60 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.791+0000 7f12f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12fc0d1570 0x7f12fc0aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.791+0000 7f12f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12fc0d1570 0x7f12fc0aaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55336/0 (socket says 172.21.15.131:55336) 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.791+0000 7f12f3e00640 1 -- 172.21.15.131:0/173560304 learned_addr learned my addr 172.21.15.131:0/173560304 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.791+0000 7f1300c00640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc0cfe60 0x7f12fc0acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.791+0000 7f12f3e00640 1 -- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc0cfe60 msgr2=0x7f12fc0acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.791+0000 7f1301600640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc14ceb0 0x7f12fc0ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.791+0000 7f12f3e00640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc0cfe60 0x7f12fc0acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.791+0000 7f12f3e00640 1 -- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc14ceb0 msgr2=0x7f12fc0ab3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.791+0000 7f12f3e00640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc14ceb0 0x7f12fc0ab3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.791+0000 7f12f3e00640 1 -- 172.21.15.131:0/173560304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc1ec3d0 con 0x7f12fc0d1570 2024-04-28T12:37:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.791+0000 7f12f3e00640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12fc0d1570 0x7f12fc0aaea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f12e400c430 tx=0x7f12e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.792+0000 7f12f9600640 1 -- 172.21.15.131:0/173560304 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4011790 con 0x7f12fc0d1570 2024-04-28T12:37:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.792+0000 7f12f9600640 1 -- 172.21.15.131:0/173560304 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12e4017070 con 0x7f12fc0d1570 2024-04-28T12:37:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.792+0000 7f12f9600640 1 -- 172.21.15.131:0/173560304 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4011bb0 con 0x7f12fc0d1570 2024-04-28T12:37:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.792+0000 7f1303600640 1 -- 172.21.15.131:0/173560304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12fc1e88d0 con 0x7f12fc0d1570 2024-04-28T12:37:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.792+0000 7f1303600640 1 -- 172.21.15.131:0/173560304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12fc1e8e50 con 0x7f12fc0d1570 2024-04-28T12:37:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.794+0000 7f12f9600640 1 -- 172.21.15.131:0/173560304 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12e401c460 con 0x7f12fc0d1570 2024-04-28T12:37:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.795+0000 7f12f9600640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f12d806c8f0 0x7f12d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.796+0000 7f12f9600640 1 -- 172.21.15.131:0/173560304 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f12e408bdc0 con 0x7f12fc0d1570 2024-04-28T12:37:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.796+0000 7f1300c00640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f12d806c8f0 0x7f12d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.796+0000 7f130638e740 1 Processor -- start 2024-04-28T12:37:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.797+0000 7f130638e740 1 -- start start 2024-04-28T12:37:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.797+0000 7f1300c00640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f12d806c8f0 0x7f12d806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12ec006fd0 tx=0x7f12ec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.797+0000 7f130638e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9a35e0 0x562d6b9a5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.797+0000 7f130638e740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9dfcb0 0x562d6b9a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.797+0000 7f130638e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 0x562d6b9ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.797+0000 7f130638e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562d6b907600 con 0x562d6b9a35e0 2024-04-28T12:37:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.797+0000 7f130638e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562d6b8a4100 con 0x562d6b9ce0a0 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.797+0000 7f130638e740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x562d6b9035c0 con 0x562d6b9dfcb0 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.798+0000 7f1300c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9a35e0 0x562d6b9a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.798+0000 7f1300c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9a35e0 0x562d6b9a5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55340/0 (socket says 172.21.15.131:55340) 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.798+0000 7f1300c00640 1 -- 172.21.15.131:0/3212836462 learned_addr learned my addr 172.21.15.131:0/3212836462 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.798+0000 7f12f3e00640 1 --2- 172.21.15.131:0/3212836462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9dfcb0 0x562d6b9a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.798+0000 7f1301600640 1 --2- 172.21.15.131:0/3212836462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 0x562d6b9ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.798+0000 7f1300c00640 1 -- 172.21.15.131:0/3212836462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9dfcb0 msgr2=0x562d6b9a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.798+0000 7f1300c00640 1 --2- 172.21.15.131:0/3212836462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9dfcb0 0x562d6b9a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.798+0000 7f1300c00640 1 -- 172.21.15.131:0/3212836462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 msgr2=0x562d6b9ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.798+0000 7f1300c00640 1 --2- 172.21.15.131:0/3212836462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 0x562d6b9ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.798+0000 7f1300c00640 1 -- 172.21.15.131:0/3212836462 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d6b6192d0 con 0x562d6b9a35e0 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.798+0000 7f12f3e00640 1 --2- 172.21.15.131:0/3212836462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9dfcb0 0x562d6b9a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.799+0000 7f1300c00640 1 --2- 172.21.15.131:0/3212836462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9a35e0 0x562d6b9a5740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00fa50 tx=0x7f12ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948445f31e09f520 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:30.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.799+0000 7f12f1200640 1 -- 172.21.15.131:0/3212836462 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec01b050 con 0x562d6b9a35e0 2024-04-28T12:37:30.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.799+0000 7f12f1200640 1 -- 172.21.15.131:0/3212836462 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ec025070 con 0x562d6b9a35e0 2024-04-28T12:37:30.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.799+0000 7f12f1200640 1 -- 172.21.15.131:0/3212836462 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec014de0 con 0x562d6b9a35e0 2024-04-28T12:37:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.800+0000 7f130638e740 1 -- 172.21.15.131:0/3212836462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9a35e0 msgr2=0x562d6b9a5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.800+0000 7f130638e740 1 --2- 172.21.15.131:0/3212836462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9a35e0 0x562d6b9a5740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00fa50 tx=0x7f12ec012000 comp rx=0 tx=0).stop 2024-04-28T12:37:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.800+0000 7f130638e740 1 -- 172.21.15.131:0/3212836462 shutdown_connections 2024-04-28T12:37:30.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.800+0000 7f130638e740 1 --2- 172.21.15.131:0/3212836462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 0x562d6b9ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.800+0000 7f130638e740 1 --2- 172.21.15.131:0/3212836462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9dfcb0 0x562d6b9a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.800+0000 7f130638e740 1 --2- 172.21.15.131:0/3212836462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9a35e0 0x562d6b9a5740 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.800+0000 7f130638e740 1 -- 172.21.15.131:0/3212836462 >> 172.21.15.131:0/3212836462 conn(0x562d6b937d10 msgr2=0x562d6b9e90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.800+0000 7f130638e740 1 -- 172.21.15.131:0/3212836462 shutdown_connections 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.800+0000 7f130638e740 1 -- 172.21.15.131:0/3212836462 wait complete. 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.801+0000 7f130638e740 1 -- start start 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.801+0000 7f130638e740 1 Processor -- start 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.802+0000 7f130638e740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6b9ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.802+0000 7f130638e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9ce0a0 0x562d6b9e5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.802+0000 7f130638e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.802+0000 7f130638e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562d6b88ef70 con 0x562d6b9ce0a0 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.802+0000 7f130638e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562d6b9cbbf0 con 0x562d6b9dfcb0 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.802+0000 7f130638e740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x562d6b9f98c0 con 0x562d6b9a35e0 2024-04-28T12:37:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.802+0000 7f130638e740 10 client.0 _ll_register_callbacks cb 0x562d6b8fd480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:37:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.802+0000 7f12f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9ce0a0 0x562d6b9e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.802+0000 7f12f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9ce0a0 0x562d6b9e5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47128/0 (socket says 172.21.15.131:47128) 2024-04-28T12:37:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.802+0000 7f12f3e00640 1 -- 172.21.15.131:0/1112933370 learned_addr learned my addr 172.21.15.131:0/1112933370 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.803+0000 7f1300c00640 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6b9ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.803+0000 7f1301600640 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.803+0000 7f12f3e00640 1 -- 172.21.15.131:0/1112933370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 msgr2=0x562d6b9ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.803+0000 7f12f3e00640 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6b9ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.803+0000 7f12f3e00640 1 -- 172.21.15.131:0/1112933370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9dfcb0 msgr2=0x562d6b9d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.803+0000 7f12f3e00640 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.803+0000 7f12f3e00640 1 -- 172.21.15.131:0/1112933370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d6b6e36b0 con 0x562d6b9ce0a0 2024-04-28T12:37:30.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.803+0000 7f12f3e00640 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9ce0a0 0x562d6b9e5150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f12e400a040 tx=0x7f12e4059fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca24630b0d57751 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.803+0000 7f12e9600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.803+0000 7f12e9600640 1 -- 172.21.15.131:0/1112933370 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4088060 con 0x562d6b9ce0a0 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f12e9600640 1 -- 172.21.15.131:0/1112933370 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f12e4089bf0 con 0x562d6b9ce0a0 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f130638e740 1 -- 172.21.15.131:0/1112933370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12ec025070 con 0x562d6b9ce0a0 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f12e9600640 1 -- 172.21.15.131:0/1112933370 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f12e4076050 con 0x562d6b9ce0a0 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f130638e740 10 client.4386 fetch_fsmap learned FSMap version 7 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f130638e740 1 -- 172.21.15.131:0/1112933370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x562d6b9d12a0 con 0x562d6b9ce0a0 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f12e9600640 1 -- 172.21.15.131:0/1112933370 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ec025070 con 0x562d6b9ce0a0 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f12e9600640 1 -- 172.21.15.131:0/1112933370 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f12e405ece0 con 0x562d6b9ce0a0 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f130638e740 10 client.4386 fetch_fsmap finished waiting for FSMap version 7 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f130638e740 10 client.4386 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f130638e740 20 client.4386 populate_metadata read hostname 'smithi131' 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f130638e740 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x562d6b9f9130 0x562d6ba06540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f130638e740 4 client.4386 mds_command: new command op to 34160 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:37:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.804+0000 7f130638e740 1 -- 172.21.15.131:0/1112933370 --> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d6b915210 con 0x562d6b9f9130 2024-04-28T12:37:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.805+0000 7f1300c00640 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x562d6b9f9130 0x562d6ba06540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.806+0000 7f1300c00640 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x562d6b9f9130 0x562d6ba06540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67d31623078244b server_cookie=8fc55ea00e7b9944 in_seq=0 out_seq=0 2024-04-28T12:37:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.806+0000 7f12e9600640 10 client.4386 ms_handle_connect on v2:172.21.15.157:6834/1937780695 2024-04-28T12:37:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f12e9600640 1 -- 172.21.15.131:0/1112933370 <== mds.0 v2:172.21.15.157:6834/1937780695 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d6b915210 con 0x562d6b9f9130 2024-04-28T12:37:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f12e9600640 10 client.4386 handle_command_reply: tid=0 2024-04-28T12:37:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f130638e740 1 client.4386 shutdown 2024-04-28T12:37:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f130638e740 1 -- 172.21.15.131:0/1112933370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9ce0a0 msgr2=0x562d6b9e5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f130638e740 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9ce0a0 0x562d6b9e5150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f12e400a040 tx=0x7f12e4059fe0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f130638e740 1 -- 172.21.15.131:0/1112933370 shutdown_connections 2024-04-28T12:37:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f130638e740 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x562d6b9f9130 0x562d6ba06540 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f130638e740 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f130638e740 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9ce0a0 0x562d6b9e5150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f12e9600640 0 client.4386 ms_handle_reset on v2:172.21.15.157:6834/1937780695 2024-04-28T12:37:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f130638e740 1 --2- 172.21.15.131:0/1112933370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6b9ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f130638e740 1 -- 172.21.15.131:0/1112933370 >> 172.21.15.131:0/1112933370 conn(0x562d6b937d10 msgr2=0x562d6b9cb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f130638e740 1 -- 172.21.15.131:0/1112933370 shutdown_connections 2024-04-28T12:37:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.809+0000 7f130638e740 1 -- 172.21.15.131:0/1112933370 wait complete. 2024-04-28T12:37:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.810+0000 7f130638e740 20 client.4386 trim_cache size 0 max 16384 2024-04-28T12:37:30.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.830+0000 7f130638e740 1 Processor -- start 2024-04-28T12:37:30.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.830+0000 7f130638e740 1 -- start start 2024-04-28T12:37:30.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f130638e740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6ba0bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f130638e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 0x562d6ba5a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f130638e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f130638e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc205dd0 con 0x562d6b9dfcb0 2024-04-28T12:37:30.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f130638e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc205230 con 0x562d6b9ce0a0 2024-04-28T12:37:30.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f130638e740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc205760 con 0x562d6b9a35e0 2024-04-28T12:37:30.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f1301600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f1301600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47142/0 (socket says 172.21.15.131:47142) 2024-04-28T12:37:30.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f1301600640 1 -- 172.21.15.131:0/2333932895 learned_addr learned my addr 172.21.15.131:0/2333932895 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:30.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f12f3e00640 1 --2- 172.21.15.131:0/2333932895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 0x562d6ba5a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f1300c00640 1 --2- 172.21.15.131:0/2333932895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6ba0bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f1301600640 1 -- 172.21.15.131:0/2333932895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 msgr2=0x562d6ba0bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:30.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f1301600640 1 --2- 172.21.15.131:0/2333932895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6ba0bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f1301600640 1 -- 172.21.15.131:0/2333932895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 msgr2=0x562d6ba5a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:30.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f1301600640 1 --2- 172.21.15.131:0/2333932895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 0x562d6ba5a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f1301600640 1 -- 172.21.15.131:0/2333932895 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d6b9cbde0 con 0x562d6b9dfcb0 2024-04-28T12:37:30.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.831+0000 7f1300c00640 1 --2- 172.21.15.131:0/2333932895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6ba0bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:30.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.832+0000 7f1301600640 1 --2- 172.21.15.131:0/2333932895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d16f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009f10 tx=0x7f12f402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf27a6568bb1939 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:30.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.832+0000 7f12ebe00640 1 -- 172.21.15.131:0/2333932895 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4036040 con 0x562d6b9dfcb0 2024-04-28T12:37:30.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.832+0000 7f12ebe00640 1 -- 172.21.15.131:0/2333932895 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f403c640 con 0x562d6b9dfcb0 2024-04-28T12:37:30.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.832+0000 7f12ebe00640 1 -- 172.21.15.131:0/2333932895 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f403ca30 con 0x562d6b9dfcb0 2024-04-28T12:37:30.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.833+0000 7f130638e740 1 -- 172.21.15.131:0/2333932895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 msgr2=0x562d6b9d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:30.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.833+0000 7f130638e740 1 --2- 172.21.15.131:0/2333932895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d16f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009f10 tx=0x7f12f402e000 comp rx=0 tx=0).stop 2024-04-28T12:37:30.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.833+0000 7f130638e740 1 -- 172.21.15.131:0/2333932895 shutdown_connections 2024-04-28T12:37:30.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.833+0000 7f130638e740 1 --2- 172.21.15.131:0/2333932895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d16f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.833+0000 7f130638e740 1 --2- 172.21.15.131:0/2333932895 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 0x562d6ba5a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.833+0000 7f130638e740 1 --2- 172.21.15.131:0/2333932895 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6ba0bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.833+0000 7f130638e740 1 -- 172.21.15.131:0/2333932895 >> 172.21.15.131:0/2333932895 conn(0x562d6b937d10 msgr2=0x562d6b9d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.833+0000 7f130638e740 1 -- 172.21.15.131:0/2333932895 shutdown_connections 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.833+0000 7f130638e740 1 -- 172.21.15.131:0/2333932895 wait complete. 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.834+0000 7f130638e740 1 -- start start 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.834+0000 7f130638e740 1 Processor -- start 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f130638e740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6ba57140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f130638e740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 0x562d6b9d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f130638e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f130638e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562d6b9f8fb0 con 0x562d6b9dfcb0 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f130638e740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x562d6b9f96c0 con 0x562d6b9ce0a0 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f130638e740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x562d6b9cf6a0 con 0x562d6b9a35e0 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f130638e740 10 client.0 _ll_register_callbacks cb 0x7f12e4055ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f1301600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f1301600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47146/0 (socket says 172.21.15.131:47146) 2024-04-28T12:37:30.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f1301600640 1 -- 172.21.15.131:0/1758412020 learned_addr learned my addr 172.21.15.131:0/1758412020 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f1300c00640 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6ba57140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f12f3e00640 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 0x562d6b9d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f1301600640 1 -- 172.21.15.131:0/1758412020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 msgr2=0x562d6ba57140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f1301600640 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6ba57140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f1301600640 1 -- 172.21.15.131:0/1758412020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 msgr2=0x562d6b9d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f1301600640 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 0x562d6b9d1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.835+0000 7f1301600640 1 -- 172.21.15.131:0/1758412020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d6b9ec870 con 0x562d6b9dfcb0 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.836+0000 7f1301600640 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d19b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f12f40093e0 tx=0x7f12f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3fdf3c2fb3f199 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.836+0000 7f12e9600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.836+0000 7f12e9600640 1 -- 172.21.15.131:0/1758412020 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4036050 con 0x562d6b9dfcb0 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.836+0000 7f12e9600640 1 -- 172.21.15.131:0/1758412020 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f12f403c9f0 con 0x562d6b9dfcb0 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.836+0000 7f130638e740 1 -- 172.21.15.131:0/1758412020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f403c640 con 0x562d6b9dfcb0 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.836+0000 7f12e9600640 1 -- 172.21.15.131:0/1758412020 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f12f4059050 con 0x562d6b9dfcb0 2024-04-28T12:37:30.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.836+0000 7f130638e740 10 client.4392 fetch_fsmap learned FSMap version 7 2024-04-28T12:37:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.836+0000 7f130638e740 1 -- 172.21.15.131:0/1758412020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x562d6ba5c700 con 0x562d6b9dfcb0 2024-04-28T12:37:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.836+0000 7f12e9600640 1 -- 172.21.15.131:0/1758412020 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f403c640 con 0x562d6b9dfcb0 2024-04-28T12:37:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.837+0000 7f12e9600640 1 -- 172.21.15.131:0/1758412020 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f12f405c020 con 0x562d6b9dfcb0 2024-04-28T12:37:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.837+0000 7f130638e740 10 client.4392 fetch_fsmap finished waiting for FSMap version 7 2024-04-28T12:37:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.837+0000 7f130638e740 10 client.4392 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T12:37:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.837+0000 7f130638e740 20 client.4392 populate_metadata read hostname 'smithi131' 2024-04-28T12:37:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.837+0000 7f130638e740 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x562d6b9df400 0x562d6ba5b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.837+0000 7f130638e740 4 client.4392 mds_command: new command op to 34160 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:37:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.837+0000 7f130638e740 1 -- 172.21.15.131:0/1758412020 --> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] -- command(tid 0: {"prefix": "status"}) -- 0x562d6b90f080 con 0x562d6b9df400 2024-04-28T12:37:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.837+0000 7f1300c00640 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x562d6b9df400 0x562d6ba5b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.838+0000 7f1300c00640 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x562d6b9df400 0x562d6ba5b680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b06344ddc9162b2 server_cookie=70c52177791d8ef6 in_seq=0 out_seq=0 2024-04-28T12:37:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.838+0000 7f12e9600640 10 client.4392 ms_handle_connect on v2:172.21.15.157:6834/1937780695 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f12e9600640 1 -- 172.21.15.131:0/1758412020 <== mds.0 v2:172.21.15.157:6834/1937780695 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x562d6b90f080 con 0x562d6b9df400 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f12e9600640 10 client.4392 handle_command_reply: tid=0 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f130638e740 1 client.4392 shutdown 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f130638e740 1 -- 172.21.15.131:0/1758412020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 msgr2=0x562d6b9d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f130638e740 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d19b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f12f40093e0 tx=0x7f12f4004ab0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f130638e740 1 -- 172.21.15.131:0/1758412020 shutdown_connections 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f130638e740 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x562d6b9df400 0x562d6ba5b680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f130638e740 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d6b9dfcb0 0x562d6b9d19b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f130638e740 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x562d6b9ce0a0 0x562d6b9d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f130638e740 1 --2- 172.21.15.131:0/1758412020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x562d6b9a35e0 0x562d6ba57140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f130638e740 1 -- 172.21.15.131:0/1758412020 >> 172.21.15.131:0/1758412020 conn(0x562d6b937d10 msgr2=0x562d6b9e2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f12e9600640 0 client.4392 ms_handle_reset on v2:172.21.15.157:6834/1937780695 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f130638e740 1 -- 172.21.15.131:0/1758412020 shutdown_connections 2024-04-28T12:37:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f130638e740 1 -- 172.21.15.131:0/1758412020 wait complete. 2024-04-28T12:37:30.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.839+0000 7f130638e740 20 client.4392 trim_cache size 0 max 16384 2024-04-28T12:37:30.840 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:37:30.840 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "fc0efa02-d382-4f6a-947e-cbe7ab2d58d1", 2024-04-28T12:37:30.840 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-04-28T12:37:30.840 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 34160, 2024-04-28T12:37:30.840 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-04-28T12:37:30.840 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-04-28T12:37:30.840 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-04-28T12:37:30.840 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 8.0472384389999991, 2024-04-28T12:37:30.840 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 7, 2024-04-28T12:37:30.840 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 24, 2024-04-28T12:37:30.841 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 24, 2024-04-28T12:37:30.841 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 15.842349917 2024-04-28T12:37:30.841 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:37:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.840+0000 7f1303600640 1 -- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f12d806c8f0 msgr2=0x7f12d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.840+0000 7f1303600640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f12d806c8f0 0x7f12d806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12ec006fd0 tx=0x7f12ec008040 comp rx=0 tx=0).stop 2024-04-28T12:37:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.840+0000 7f1303600640 1 -- 172.21.15.131:0/173560304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12fc0d1570 msgr2=0x7f12fc0aaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.840+0000 7f1303600640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12fc0d1570 0x7f12fc0aaea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f12e400c430 tx=0x7f12e400c8f0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.840+0000 7f1303600640 1 -- 172.21.15.131:0/173560304 shutdown_connections 2024-04-28T12:37:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.840+0000 7f1303600640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f12d806c8f0 0x7f12d806edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.840+0000 7f1303600640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc14ceb0 0x7f12fc0ab3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.840+0000 7f1303600640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12fc0d1570 0x7f12fc0aaea0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.840+0000 7f1303600640 1 --2- 172.21.15.131:0/173560304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12fc0cfe60 0x7f12fc0acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.840+0000 7f1303600640 1 -- 172.21.15.131:0/173560304 >> 172.21.15.131:0/173560304 conn(0x7f12fc0b2900 msgr2=0x7f12fc0ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.840+0000 7f1303600640 1 -- 172.21.15.131:0/173560304 shutdown_connections 2024-04-28T12:37:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:30.840+0000 7f1303600640 1 -- 172.21.15.131:0/173560304 wait complete. 2024-04-28T12:37:30.855 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-28T12:37:30.856 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:37:30.856 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T12:37:31.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.055+0000 7fb8fa400640 1 Processor -- start 2024-04-28T12:37:31.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.055+0000 7fb8fa400640 1 -- start start 2024-04-28T12:37:31.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.057+0000 7fb8fa400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb8f415b900 0x7fb8f40b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:31.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.057+0000 7fb8fa400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb8f406f840 0x7fb8f415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:31.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.057+0000 7fb8fa400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8f4071d20 con 0x7fb8f415fce0 2024-04-28T12:37:31.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.057+0000 7fb8fa400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb8f4160130 con 0x7fb8f406f840 2024-04-28T12:37:31.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.057+0000 7fb8fa400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb8f40b4f00 con 0x7fb8f415b900 2024-04-28T12:37:31.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.057+0000 7fb8f3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb8f406f840 0x7fb8f415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:31.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.057+0000 7fb8f9a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb8f415fce0 0x7fb8f40724b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33720/0 (socket says 172.21.15.131:33720) 2024-04-28T12:37:31.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.057+0000 7fb8f9a00640 1 -- 172.21.15.131:0/719081440 learned_addr learned my addr 172.21.15.131:0/719081440 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:31.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.057+0000 7fb8f3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb8f406f840 0x7fb8f415f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47148/0 (socket says 172.21.15.131:47148) 2024-04-28T12:37:31.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.057+0000 7fb8f9000640 1 --2- 172.21.15.131:0/719081440 >> v2:172.21.15.157:3300/0 conn(0x7fb8f415b900 0x7fb8f40b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:31.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.058+0000 7fb8f3e00640 1 -- 172.21.15.131:0/719081440 >> v2:172.21.15.157:3300/0 conn(0x7fb8f415b900 msgr2=0x7fb8f40b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:31.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.058+0000 7fb8f3e00640 1 --2- 172.21.15.131:0/719081440 >> v2:172.21.15.157:3300/0 conn(0x7fb8f415b900 0x7fb8f40b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:31.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.058+0000 7fb8f3e00640 1 -- 172.21.15.131:0/719081440 >> v1:172.21.15.131:6789/0 conn(0x7fb8f415fce0 legacy=0x7fb8f40724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:31.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.058+0000 7fb8f3e00640 1 -- 172.21.15.131:0/719081440 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f40b51e0 con 0x7fb8f406f840 2024-04-28T12:37:31.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.058+0000 7fb8f3400640 1 -- 172.21.15.131:0/719081440 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2298460753 0 0) 0x7fb8f4071d20 con 0x7fb8f415fce0 2024-04-28T12:37:31.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.059+0000 7fb8f3e00640 1 --2- 172.21.15.131:0/719081440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8f406f840 0x7fb8f415f7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40b32c0 tx=0x7fb8e002c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ff8d9617557635 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:31.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.059+0000 7fb8f3400640 1 -- 172.21.15.131:0/719081440 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0032770 con 0x7fb8f406f840 2024-04-28T12:37:31.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.059+0000 7fb8f3400640 1 -- 172.21.15.131:0/719081440 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e0038070 con 0x7fb8f406f840 2024-04-28T12:37:31.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.059+0000 7fb8f3400640 1 -- 172.21.15.131:0/719081440 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0032b30 con 0x7fb8f406f840 2024-04-28T12:37:31.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.060+0000 7fb8fa400640 1 -- 172.21.15.131:0/719081440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8f406f840 msgr2=0x7fb8f415f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:31.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.060+0000 7fb8fa400640 1 --2- 172.21.15.131:0/719081440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8f406f840 0x7fb8f415f7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40b32c0 tx=0x7fb8e002c870 comp rx=0 tx=0).stop 2024-04-28T12:37:31.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.060+0000 7fb8fa400640 1 -- 172.21.15.131:0/719081440 shutdown_connections 2024-04-28T12:37:31.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.060+0000 7fb8fa400640 1 --2- 172.21.15.131:0/719081440 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8f406f840 0x7fb8f415f7a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:31.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.060+0000 7fb8fa400640 1 --2- 172.21.15.131:0/719081440 >> v2:172.21.15.157:3300/0 conn(0x7fb8f415b900 0x7fb8f40b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:31.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.060+0000 7fb8fa400640 1 -- 172.21.15.131:0/719081440 >> 172.21.15.131:0/719081440 conn(0x7fb8f40b26b0 msgr2=0x7fb8f40b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:31.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.060+0000 7fb8fa400640 1 -- 172.21.15.131:0/719081440 shutdown_connections 2024-04-28T12:37:31.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.060+0000 7fb8fa400640 1 -- 172.21.15.131:0/719081440 wait complete. 2024-04-28T12:37:31.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.062+0000 7fb8fa400640 1 Processor -- start 2024-04-28T12:37:31.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.062+0000 7fb8fa400640 1 -- start start 2024-04-28T12:37:31.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8fa400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8f406f840 0x7fb8f40cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:31.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8fa400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f415b900 0x7fb8f4158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8fa400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8f415fce0 0x7fb8f414b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8fa400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f40b54c0 con 0x7fb8f406f840 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8fa400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f40b4c10 con 0x7fb8f415fce0 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8fa400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f40b5640 con 0x7fb8f415b900 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8f9000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8f406f840 0x7fb8f40cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8f9000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8f406f840 0x7fb8f40cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47152/0 (socket says 172.21.15.131:47152) 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8f9000640 1 -- 172.21.15.131:0/2910313990 learned_addr learned my addr 172.21.15.131:0/2910313990 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8f9a00640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8f415fce0 0x7fb8f414b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8f9000640 1 -- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f415b900 msgr2=0x7fb8f4158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8f9000640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f415b900 0x7fb8f4158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8f9000640 1 -- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8f415fce0 msgr2=0x7fb8f414b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8f9000640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8f415fce0 0x7fb8f414b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.063+0000 7fb8f9000640 1 -- 172.21.15.131:0/2910313990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f4152a70 con 0x7fb8f406f840 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.064+0000 7fb8f9000640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8f406f840 0x7fb8f40cf390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800e9b0 tx=0x7fb8e800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.064+0000 7fb8f0c00640 1 -- 172.21.15.131:0/2910313990 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8017070 con 0x7fb8f406f840 2024-04-28T12:37:31.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.064+0000 7fb8f0c00640 1 -- 172.21.15.131:0/2910313990 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e800c9a0 con 0x7fb8f406f840 2024-04-28T12:37:31.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.064+0000 7fb8f0c00640 1 -- 172.21.15.131:0/2910313990 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e800cd90 con 0x7fb8f406f840 2024-04-28T12:37:31.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.064+0000 7fb8fa400640 1 -- 172.21.15.131:0/2910313990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f4152d50 con 0x7fb8f406f840 2024-04-28T12:37:31.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.064+0000 7fb8fa400640 1 -- 172.21.15.131:0/2910313990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f4153310 con 0x7fb8f406f840 2024-04-28T12:37:31.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.065+0000 7fb8fa400640 1 -- 172.21.15.131:0/2910313990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f406e4e0 con 0x7fb8f406f840 2024-04-28T12:37:31.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.066+0000 7fb8f0c00640 1 -- 172.21.15.131:0/2910313990 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8e801a070 con 0x7fb8f406f840 2024-04-28T12:37:31.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.067+0000 7fb8f0c00640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb8d406c8f0 0x7fb8d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:31.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.067+0000 7fb8f0c00640 1 -- 172.21.15.131:0/2910313990 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb8e808be00 con 0x7fb8f406f840 2024-04-28T12:37:31.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.068+0000 7fb8f3e00640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb8d406c8f0 0x7fb8d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:31.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.069+0000 7fb8f3e00640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb8d406c8f0 0x7fb8d406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0037000 tx=0x7fb8e0031000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:31.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.071+0000 7fb8f0c00640 1 -- 172.21.15.131:0/2910313990 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8f406e4e0 con 0x7fb8f406f840 2024-04-28T12:37:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.310+0000 7fb8fa400640 1 -- 172.21.15.131:0/2910313990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb8f4069350 con 0x7fb8f406f840 2024-04-28T12:37:31.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.312+0000 7fb8f0c00640 1 -- 172.21.15.131:0/2910313990 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7fb8e8059f80 con 0x7fb8f406f840 2024-04-28T12:37:31.309 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-28T12:37:31.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.315+0000 7fb8fa400640 1 -- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb8d406c8f0 msgr2=0x7fb8d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:31.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.315+0000 7fb8fa400640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb8d406c8f0 0x7fb8d406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0037000 tx=0x7fb8e0031000 comp rx=0 tx=0).stop 2024-04-28T12:37:31.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.315+0000 7fb8fa400640 1 -- 172.21.15.131:0/2910313990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8f406f840 msgr2=0x7fb8f40cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:31.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.315+0000 7fb8fa400640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8f406f840 0x7fb8f40cf390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800e9b0 tx=0x7fb8e800ee70 comp rx=0 tx=0).stop 2024-04-28T12:37:31.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.315+0000 7fb8fa400640 1 -- 172.21.15.131:0/2910313990 shutdown_connections 2024-04-28T12:37:31.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.315+0000 7fb8fa400640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb8d406c8f0 0x7fb8d406edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:31.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.315+0000 7fb8fa400640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8f415fce0 0x7fb8f414b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:31.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.315+0000 7fb8fa400640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8f415b900 0x7fb8f4158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:31.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.315+0000 7fb8fa400640 1 --2- 172.21.15.131:0/2910313990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8f406f840 0x7fb8f40cf390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:31.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.315+0000 7fb8fa400640 1 -- 172.21.15.131:0/2910313990 >> 172.21.15.131:0/2910313990 conn(0x7fb8f40b26b0 msgr2=0x7fb8f406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:31.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.315+0000 7fb8fa400640 1 -- 172.21.15.131:0/2910313990 shutdown_connections 2024-04-28T12:37:31.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31.315+0000 7fb8fa400640 1 -- 172.21.15.131:0/2910313990 wait complete. 2024-04-28T12:37:31.330 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-28T12:37:31.330 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T12:37:31.330 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-04-28T12:37:31.331 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-28T12:37:31.331 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:37:31.331 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-28T12:37:31.331 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-28T12:37:31.331 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T12:37:31.331 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-28T12:37:31.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:31.331 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-04-28T12:37:31.380 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T12:37:31.380 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T12:37:31.380 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T12:37:31.380 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:37:31.380 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-04-28T12:37:31.380 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42524sec preferred_lft 42524sec 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-04-28T12:37:31.381 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:37:31.383 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-28T12:37:31.383 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:37:31.383 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:37:31.383 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add name ceph-brx type bridge 2024-04-28T12:37:31.383 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr flush dev ceph-brx 2024-04-28T12:37:31.383 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set ceph-brx up 2024-04-28T12:37:31.383 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-28T12:37:31.383 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:37:31.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:37:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:37:31.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:31.878 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T12:37:31.945 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-04-28T12:37:31.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:31.948 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-04-28T12:37:32.012 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-04-28T12:37:32.013 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T12:37:32.013 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T12:37:32.013 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T12:37:32.013 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T12:37:32.013 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:37:32.014 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:37:32.014 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T12:37:32.014 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T12:37:32.014 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T12:37:32.014 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:37:32.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:37:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:37:32.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:32.762 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-04-28T12:37:32.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:32.808 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-04-28T12:37:32.891 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:37:32.891 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:37:32.891 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:37:32.891 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-28T12:37:32.891 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:37:33.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:37:33.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:37:33.102 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-28T12:37:33.102 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:37:33.102 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:37:33.102 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-28T12:37:33.102 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-28T12:37:33.102 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-28T12:37:33.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-28T12:37:33.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-28T12:37:33.103 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:37:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:37:33.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:37:33.369 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:37:33.369 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:37:33.369 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-04-28T12:37:33.369 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.0 master ceph-brx 2024-04-28T12:37:33.369 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:37:33.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:37:33.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:37:33.555 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-28T12:37:33.555 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:37:33.556 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:37:33.595 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-28T12:37:33.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:37:33.595 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:37:33.683 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-04-28T12:37:33.787 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:37:33.831 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-04-28T12:37:33.831 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-04-28T12:37:33.831 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/selinux 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-04-28T12:37:33.832 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:37:33.833 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:37:33.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:33.834 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-04-28T12:37:33.927 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-28T12:37:33.927 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-28T12:37:33.932 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-04-28T12:37:33.998 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:37:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-04-28T12:37:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-04-28T12:37:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-04-28T12:37:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-04-28T12:37:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-04-28T12:37:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-04-28T12:37:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-04-28T12:37:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-04-28T12:37:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-04-28T12:37:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-04-28T12:37:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-04-28T12:37:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/selinux 2024-04-28T12:37:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:37:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-04-28T12:37:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-04-28T12:37:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-04-28T12:37:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:37:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:37:34.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:34.042 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-04-28T12:37:34.110 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:2024-04-28T12:37:34.114+0000 7f10c740c480 -1 init, newargv = 0x56050f74fc80 newargc=15 2024-04-28T12:37:34.110 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:ceph-fuse[39892]: starting ceph client 2024-04-28T12:37:34.124 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:ceph-fuse[39892]: starting fuse 2024-04-28T12:37:35.224 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-04-28T12:37:35.308 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:37:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-04-28T12:37:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-04-28T12:37:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-04-28T12:37:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-04-28T12:37:35.360 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/selinux 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-04-28T12:37:35.361 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:37:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-28T12:37:35.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:35.364 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-04-28T12:37:35.410 INFO:teuthology.orchestra.run.smithi131.stdout:41 2024-04-28T12:37:35.410 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-28T12:37:35.410 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-04-28T12:37:35.410 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-04-28T12:37:35.410 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-04-28T12:37:35.410 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:37:35.411 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-04-28T12:37:35.412 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-28T12:37:35.412 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-28T12:37:35.412 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-28T12:37:35.412 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-04-28T12:37:35.412 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-04-28T12:37:35.412 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-04-28T12:37:35.412 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-28T12:37:35.412 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:37:35.412 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.admin")) 2024-04-28T12:37:35.412 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-04-28T12:37:35.621 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.admin.39892.asok 2024-04-28T12:37:35.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:37:35.640 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.39892.asok 2024-04-28T12:37:35.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:35.640 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.39892.asok status 2024-04-28T12:37:35.861 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:37:35.861 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-04-28T12:37:35.861 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-28T12:37:35.861 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-28T12:37:35.861 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "admin", 2024-04-28T12:37:35.861 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-04-28T12:37:35.861 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-28T12:37:35.861 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "39892", 2024-04-28T12:37:35.862 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-04-28T12:37:35.862 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:37:35.862 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-04-28T12:37:35.862 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-04-28T12:37:35.862 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4410, 2024-04-28T12:37:35.862 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-04-28T12:37:35.862 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-04-28T12:37:35.862 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-04-28T12:37:35.862 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 4410 2024-04-28T12:37:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:37:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-04-28T12:37:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-04-28T12:37:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-04-28T12:37:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3823136112 2024-04-28T12:37:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:37:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:37:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-04-28T12:37:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-04-28T12:37:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-04-28T12:37:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3823136112 2024-04-28T12:37:35.863 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:37:35.864 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.4410 192.168.0.1:0/3823136112", 2024-04-28T12:37:35.864 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/3823136112", 2024-04-28T12:37:35.864 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-04-28T12:37:35.864 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 7, 2024-04-28T12:37:35.864 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 24, 2024-04-28T12:37:35.864 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-04-28T12:37:35.864 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-04-28T12:37:35.864 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-04-28T12:37:35.864 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:37:35.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:35.876 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-04-28T12:37:35.876 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:37:35.930 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-04-28T12:37:35.930 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:37:35.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:35.933 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:37:36.017 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-28T12:37:36.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T12:37:36.171 INFO:teuthology.orchestra.run.smithi131.stdout:# file: . 2024-04-28T12:37:36.172 INFO:teuthology.orchestra.run.smithi131.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-28T12:37:36.172 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:36.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:37:36.182 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:37:36.182 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:37:36.231 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:36.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.437+0000 7ff5f0200640 1 Processor -- start 2024-04-28T12:37:36.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.437+0000 7ff5f0200640 1 -- start start 2024-04-28T12:37:36.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.438+0000 7ff5f0200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff5e8145590 0x7ff5e80ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:36.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.439+0000 7ff5f0200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff5e80b3490 0x7ff5e806f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:36.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.439+0000 7ff5f0200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e80cf060 con 0x7ff5e806f560 2024-04-28T12:37:36.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.439+0000 7ff5f0200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e80d0100 con 0x7ff5e80b3490 2024-04-28T12:37:36.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.439+0000 7ff5f0200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e806f990 con 0x7ff5e8145590 2024-04-28T12:37:36.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.439+0000 7ff5ece00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff5e80b3490 0x7ff5e806f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:36.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.439+0000 7ff5ece00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff5e80b3490 0x7ff5e806f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47154/0 (socket says 172.21.15.131:47154) 2024-04-28T12:37:36.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.439+0000 7ff5ece00640 1 -- 172.21.15.131:0/2663369280 learned_addr learned my addr 172.21.15.131:0/2663369280 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:36.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.439+0000 7ff5ed800640 1 --2- 172.21.15.131:0/2663369280 >> v2:172.21.15.157:3300/0 conn(0x7ff5e8145590 0x7ff5e80ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:36.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.440+0000 7ff5ece00640 1 -- 172.21.15.131:0/2663369280 >> v2:172.21.15.157:3300/0 conn(0x7ff5e8145590 msgr2=0x7ff5e80ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:36.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.440+0000 7ff5ece00640 1 --2- 172.21.15.131:0/2663369280 >> v2:172.21.15.157:3300/0 conn(0x7ff5e8145590 0x7ff5e80ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.440+0000 7ff5ece00640 1 -- 172.21.15.131:0/2663369280 >> v1:172.21.15.157:6789/0 conn(0x7ff5e806f560 legacy=0x7ff5e80d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:36.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.440+0000 7ff5ece00640 1 -- 172.21.15.131:0/2663369280 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e80cf340 con 0x7ff5e80b3490 2024-04-28T12:37:36.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.441+0000 7ff5ece00640 1 --2- 172.21.15.131:0/2663369280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e80b3490 0x7ff5e806f020 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80ce730 tx=0x7ff5d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e837bbd1a7b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:36.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.441+0000 7ff5e7e00640 1 -- 172.21.15.131:0/2663369280 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4037070 con 0x7ff5e80b3490 2024-04-28T12:37:36.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.441+0000 7ff5e7e00640 1 -- 172.21.15.131:0/2663369280 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d4032440 con 0x7ff5e80b3490 2024-04-28T12:37:36.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.441+0000 7ff5e7e00640 1 -- 172.21.15.131:0/2663369280 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4032810 con 0x7ff5e80b3490 2024-04-28T12:37:36.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.442+0000 7ff5f0200640 1 -- 172.21.15.131:0/2663369280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e80b3490 msgr2=0x7ff5e806f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:36.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.442+0000 7ff5f0200640 1 --2- 172.21.15.131:0/2663369280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e80b3490 0x7ff5e806f020 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80ce730 tx=0x7ff5d402a270 comp rx=0 tx=0).stop 2024-04-28T12:37:36.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.442+0000 7ff5f0200640 1 -- 172.21.15.131:0/2663369280 shutdown_connections 2024-04-28T12:37:36.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.442+0000 7ff5f0200640 1 --2- 172.21.15.131:0/2663369280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e80b3490 0x7ff5e806f020 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.442+0000 7ff5f0200640 1 --2- 172.21.15.131:0/2663369280 >> v2:172.21.15.157:3300/0 conn(0x7ff5e8145590 0x7ff5e80ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.442+0000 7ff5f0200640 1 -- 172.21.15.131:0/2663369280 >> 172.21.15.131:0/2663369280 conn(0x7ff5e80b2900 msgr2=0x7ff5e80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:36.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.442+0000 7ff5f0200640 1 -- 172.21.15.131:0/2663369280 shutdown_connections 2024-04-28T12:37:36.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.442+0000 7ff5f0200640 1 -- 172.21.15.131:0/2663369280 wait complete. 2024-04-28T12:37:36.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.444+0000 7ff5f0200640 1 Processor -- start 2024-04-28T12:37:36.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.444+0000 7ff5f0200640 1 -- start start 2024-04-28T12:37:36.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.445+0000 7ff5f0200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e806f560 0x7ff5e8133310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:36.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.445+0000 7ff5f0200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e80b3490 0x7ff5e80723e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:36.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.445+0000 7ff5f0200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5e8145590 0x7ff5e8072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:36.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.445+0000 7ff5f0200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e80cfbe0 con 0x7ff5e806f560 2024-04-28T12:37:36.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.445+0000 7ff5f0200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e80cf900 con 0x7ff5e80b3490 2024-04-28T12:37:36.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.445+0000 7ff5f0200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e8145970 con 0x7ff5e8145590 2024-04-28T12:37:36.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.445+0000 7ff5ed800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e806f560 0x7ff5e8133310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:36.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.445+0000 7ff5ed800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e806f560 0x7ff5e8133310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47168/0 (socket says 172.21.15.131:47168) 2024-04-28T12:37:36.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.445+0000 7ff5ed800640 1 -- 172.21.15.131:0/2829169641 learned_addr learned my addr 172.21.15.131:0/2829169641 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:36.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.445+0000 7ff5ece00640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e80b3490 0x7ff5e80723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:36.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.446+0000 7ff5ee200640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5e8145590 0x7ff5e8072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:36.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.446+0000 7ff5ed800640 1 -- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5e8145590 msgr2=0x7ff5e8072920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:36.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.446+0000 7ff5ed800640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5e8145590 0x7ff5e8072920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.446+0000 7ff5ed800640 1 -- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e80b3490 msgr2=0x7ff5e80723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:36.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.446+0000 7ff5ed800640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e80b3490 0x7ff5e80723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.446+0000 7ff5ed800640 1 -- 172.21.15.131:0/2829169641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e8133d90 con 0x7ff5e806f560 2024-04-28T12:37:36.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.446+0000 7ff5ed800640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e806f560 0x7ff5e8133310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc006fd0 tx=0x7ff5dc008700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:36.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.446+0000 7ff5e5600640 1 -- 172.21.15.131:0/2829169641 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc01b070 con 0x7ff5e806f560 2024-04-28T12:37:36.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.446+0000 7ff5e5600640 1 -- 172.21.15.131:0/2829169641 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5dc009b10 con 0x7ff5e806f560 2024-04-28T12:37:36.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.446+0000 7ff5e5600640 1 -- 172.21.15.131:0/2829169641 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc0163f0 con 0x7ff5e806f560 2024-04-28T12:37:36.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.446+0000 7ff5f0200640 1 -- 172.21.15.131:0/2829169641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e8134070 con 0x7ff5e806f560 2024-04-28T12:37:36.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.446+0000 7ff5f0200640 1 -- 172.21.15.131:0/2829169641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e80b0280 con 0x7ff5e806f560 2024-04-28T12:37:36.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.448+0000 7ff5f0200640 1 -- 172.21.15.131:0/2829169641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e8069350 con 0x7ff5e806f560 2024-04-28T12:37:36.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.449+0000 7ff5e5600640 1 -- 172.21.15.131:0/2829169641 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5dc01e070 con 0x7ff5e806f560 2024-04-28T12:37:36.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.449+0000 7ff5e5600640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff5c806c8f0 0x7ff5c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:36.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.450+0000 7ff5e5600640 1 -- 172.21.15.131:0/2829169641 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff5dc08bb50 con 0x7ff5e806f560 2024-04-28T12:37:36.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.450+0000 7ff5ece00640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff5c806c8f0 0x7ff5c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:36.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.451+0000 7ff5ece00640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff5c806c8f0 0x7ff5c806edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4034000 tx=0x7ff5d402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:36.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.455+0000 7ff5e5600640 1 -- 172.21.15.131:0/2829169641 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5dc059550 con 0x7ff5e806f560 2024-04-28T12:37:36.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.657+0000 7ff5f0200640 1 -- 172.21.15.131:0/2829169641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5e80ce9e0 con 0x7ff5e806f560 2024-04-28T12:37:36.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.659+0000 7ff5e5600640 1 -- 172.21.15.131:0/2829169641 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7ff5dc05d850 con 0x7ff5e806f560 2024-04-28T12:37:36.656 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:36.656 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:36.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.662+0000 7ff5f0200640 1 -- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff5c806c8f0 msgr2=0x7ff5c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:36.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.662+0000 7ff5f0200640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff5c806c8f0 0x7ff5c806edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4034000 tx=0x7ff5d402e000 comp rx=0 tx=0).stop 2024-04-28T12:37:36.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.662+0000 7ff5f0200640 1 -- 172.21.15.131:0/2829169641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e806f560 msgr2=0x7ff5e8133310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:36.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.662+0000 7ff5f0200640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e806f560 0x7ff5e8133310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc006fd0 tx=0x7ff5dc008700 comp rx=0 tx=0).stop 2024-04-28T12:37:36.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.662+0000 7ff5f0200640 1 -- 172.21.15.131:0/2829169641 shutdown_connections 2024-04-28T12:37:36.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.662+0000 7ff5f0200640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff5c806c8f0 0x7ff5c806edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.662+0000 7ff5f0200640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5e8145590 0x7ff5e8072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.662+0000 7ff5f0200640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5e80b3490 0x7ff5e80723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.662+0000 7ff5f0200640 1 --2- 172.21.15.131:0/2829169641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff5e806f560 0x7ff5e8133310 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.662+0000 7ff5f0200640 1 -- 172.21.15.131:0/2829169641 >> 172.21.15.131:0/2829169641 conn(0x7ff5e80b2900 msgr2=0x7ff5e80b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:36.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.662+0000 7ff5f0200640 1 -- 172.21.15.131:0/2829169641 shutdown_connections 2024-04-28T12:37:36.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.662+0000 7ff5f0200640 1 -- 172.21.15.131:0/2829169641 wait complete. 2024-04-28T12:37:36.679 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:37:36.679 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:37:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.883+0000 7fe672000640 1 Processor -- start 2024-04-28T12:37:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.884+0000 7fe672000640 1 -- start start 2024-04-28T12:37:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.885+0000 7fe672000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe66c15f280 0x7fe66c06de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.885+0000 7fe672000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe66c0ce660 0x7fe66c06e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.885+0000 7fe672000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c0d1ba0 con 0x7fe66c15f660 2024-04-28T12:37:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.885+0000 7fe672000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe66c0d2470 con 0x7fe66c0ce660 2024-04-28T12:37:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.885+0000 7fe672000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe66c06da50 con 0x7fe66c15f280 2024-04-28T12:37:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.886+0000 7fe66be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe66c0ce660 0x7fe66c06e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.886+0000 7fe66be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe66c0ce660 0x7fe66c06e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47170/0 (socket says 172.21.15.131:47170) 2024-04-28T12:37:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.886+0000 7fe66be00640 1 -- 172.21.15.131:0/2597209041 learned_addr learned my addr 172.21.15.131:0/2597209041 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.886+0000 7fe66b400640 1 --2- 172.21.15.131:0/2597209041 >> v2:172.21.15.157:3300/0 conn(0x7fe66c15f280 0x7fe66c06de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.886+0000 7fe66be00640 1 -- 172.21.15.131:0/2597209041 >> v2:172.21.15.157:3300/0 conn(0x7fe66c15f280 msgr2=0x7fe66c06de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.886+0000 7fe66be00640 1 --2- 172.21.15.131:0/2597209041 >> v2:172.21.15.157:3300/0 conn(0x7fe66c15f280 0x7fe66c06de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.886+0000 7fe66be00640 1 -- 172.21.15.131:0/2597209041 >> v1:172.21.15.157:6789/0 conn(0x7fe66c15f660 legacy=0x7fe66c163080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.886+0000 7fe66be00640 1 -- 172.21.15.131:0/2597209041 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c0d1e20 con 0x7fe66c0ce660 2024-04-28T12:37:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.887+0000 7fe66be00640 1 --2- 172.21.15.131:0/2597209041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe66c0ce660 0x7fe66c06e3b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe66c072f00 tx=0x7fe66402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460a15dee81cbde7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.887+0000 7fe66a000640 1 -- 172.21.15.131:0/2597209041 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664036070 con 0x7fe66c0ce660 2024-04-28T12:37:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.887+0000 7fe66a000640 1 -- 172.21.15.131:0/2597209041 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe664031600 con 0x7fe66c0ce660 2024-04-28T12:37:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.887+0000 7fe66a000640 1 -- 172.21.15.131:0/2597209041 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6640319d0 con 0x7fe66c0ce660 2024-04-28T12:37:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.888+0000 7fe672000640 1 -- 172.21.15.131:0/2597209041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe66c0ce660 msgr2=0x7fe66c06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.888+0000 7fe672000640 1 --2- 172.21.15.131:0/2597209041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe66c0ce660 0x7fe66c06e3b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe66c072f00 tx=0x7fe66402a660 comp rx=0 tx=0).stop 2024-04-28T12:37:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.888+0000 7fe672000640 1 -- 172.21.15.131:0/2597209041 shutdown_connections 2024-04-28T12:37:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.888+0000 7fe672000640 1 --2- 172.21.15.131:0/2597209041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe66c0ce660 0x7fe66c06e3b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.888+0000 7fe672000640 1 --2- 172.21.15.131:0/2597209041 >> v2:172.21.15.157:3300/0 conn(0x7fe66c15f280 0x7fe66c06de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.888+0000 7fe672000640 1 -- 172.21.15.131:0/2597209041 >> 172.21.15.131:0/2597209041 conn(0x7fe66c0d2d90 msgr2=0x7fe66c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:36.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.888+0000 7fe672000640 1 -- 172.21.15.131:0/2597209041 shutdown_connections 2024-04-28T12:37:36.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.888+0000 7fe672000640 1 -- 172.21.15.131:0/2597209041 wait complete. 2024-04-28T12:37:36.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.890+0000 7fe672000640 1 Processor -- start 2024-04-28T12:37:36.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.890+0000 7fe672000640 1 -- start start 2024-04-28T12:37:36.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.891+0000 7fe672000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe66c0ce660 0x7fe66c1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:36.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.891+0000 7fe672000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe66c15f280 0x7fe66c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.891+0000 7fe672000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe66c15f660 0x7fe66c071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.891+0000 7fe672000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c06d7f0 con 0x7fe66c15f660 2024-04-28T12:37:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.891+0000 7fe672000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c06d670 con 0x7fe66c0ce660 2024-04-28T12:37:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.891+0000 7fe672000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c0d29c0 con 0x7fe66c15f280 2024-04-28T12:37:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.891+0000 7fe66be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe66c15f660 0x7fe66c071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.891+0000 7fe66be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe66c15f660 0x7fe66c071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47172/0 (socket says 172.21.15.131:47172) 2024-04-28T12:37:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.891+0000 7fe66be00640 1 -- 172.21.15.131:0/2203551615 learned_addr learned my addr 172.21.15.131:0/2203551615 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe66b400640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe66c0ce660 0x7fe66c1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe66aa00640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe66c15f280 0x7fe66c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe66be00640 1 -- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe66c15f280 msgr2=0x7fe66c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe66be00640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe66c15f280 0x7fe66c070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe66be00640 1 -- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe66c0ce660 msgr2=0x7fe66c1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe66be00640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe66c0ce660 0x7fe66c1561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe66be00640 1 -- 172.21.15.131:0/2203551615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c0c80e0 con 0x7fe66c15f660 2024-04-28T12:37:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe66aa00640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe66c15f280 0x7fe66c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe66b400640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe66c0ce660 0x7fe66c1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe66be00640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe66c15f660 0x7fe66c071530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe664009f40 tx=0x7fe664004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:36.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe663400640 1 -- 172.21.15.131:0/2203551615 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664036050 con 0x7fe66c15f660 2024-04-28T12:37:36.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe663400640 1 -- 172.21.15.131:0/2203551615 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe664042070 con 0x7fe66c15f660 2024-04-28T12:37:36.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe663400640 1 -- 172.21.15.131:0/2203551615 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664009510 con 0x7fe66c15f660 2024-04-28T12:37:36.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.892+0000 7fe672000640 1 -- 172.21.15.131:0/2203551615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c156710 con 0x7fe66c15f660 2024-04-28T12:37:36.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.893+0000 7fe672000640 1 -- 172.21.15.131:0/2203551615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c0cbff0 con 0x7fe66c15f660 2024-04-28T12:37:36.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.894+0000 7fe672000640 1 -- 172.21.15.131:0/2203551615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66c069a60 con 0x7fe66c15f660 2024-04-28T12:37:36.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.895+0000 7fe663400640 1 -- 172.21.15.131:0/2203551615 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe664007bd0 con 0x7fe66c15f660 2024-04-28T12:37:36.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.896+0000 7fe663400640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe64806c8f0 0x7fe64806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:36.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.896+0000 7fe663400640 1 -- 172.21.15.131:0/2203551615 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe6640ace50 con 0x7fe66c15f660 2024-04-28T12:37:36.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.897+0000 7fe66b400640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe64806c8f0 0x7fe64806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:36.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.898+0000 7fe66b400640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe64806c8f0 0x7fe64806edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe65c006fd0 tx=0x7fe65c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:36.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:36.901+0000 7fe663400640 1 -- 172.21.15.131:0/2203551615 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe66c0d1ba0 con 0x7fe66c15f660 2024-04-28T12:37:37.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.101+0000 7fe672000640 1 -- 172.21.15.131:0/2203551615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe66c15e6c0 con 0x7fe66c15f660 2024-04-28T12:37:37.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.102+0000 7fe663400640 1 -- 172.21.15.131:0/2203551615 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fe664045360 con 0x7fe66c15f660 2024-04-28T12:37:37.099 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:37:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.104+0000 7fe672000640 1 -- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe64806c8f0 msgr2=0x7fe64806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.104+0000 7fe672000640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe64806c8f0 0x7fe64806edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe65c006fd0 tx=0x7fe65c008040 comp rx=0 tx=0).stop 2024-04-28T12:37:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.104+0000 7fe672000640 1 -- 172.21.15.131:0/2203551615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe66c15f660 msgr2=0x7fe66c071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.104+0000 7fe672000640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe66c15f660 0x7fe66c071530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe664009f40 tx=0x7fe664004a40 comp rx=0 tx=0).stop 2024-04-28T12:37:37.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.104+0000 7fe672000640 1 -- 172.21.15.131:0/2203551615 shutdown_connections 2024-04-28T12:37:37.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.104+0000 7fe672000640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe64806c8f0 0x7fe64806edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.104+0000 7fe672000640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe66c15f660 0x7fe66c071530 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.104+0000 7fe672000640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe66c15f280 0x7fe66c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.105+0000 7fe672000640 1 --2- 172.21.15.131:0/2203551615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe66c0ce660 0x7fe66c1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.105+0000 7fe672000640 1 -- 172.21.15.131:0/2203551615 >> 172.21.15.131:0/2203551615 conn(0x7fe66c0d2d90 msgr2=0x7fe66c162750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:37.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.105+0000 7fe672000640 1 -- 172.21.15.131:0/2203551615 shutdown_connections 2024-04-28T12:37:37.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.105+0000 7fe672000640 1 -- 172.21.15.131:0/2203551615 wait complete. 2024-04-28T12:37:37.121 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:37:37.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.324+0000 7f2356200640 1 Processor -- start 2024-04-28T12:37:37.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.325+0000 7f2356200640 1 -- start start 2024-04-28T12:37:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.326+0000 7f2356200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f23500d3830 0x7f23500d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.326+0000 7f2356200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23500d2d70 con 0x7f23500b3cb0 2024-04-28T12:37:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.326+0000 7f2356200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23500d06e0 con 0x7f235015f180 2024-04-28T12:37:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.326+0000 7f2356200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f235015f560 con 0x7f23500d3830 2024-04-28T12:37:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.327+0000 7f234fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f23500d3830 0x7f23500d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.327+0000 7f2355800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f23500b3cb0 0x7f23500b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33726/0 (socket says 172.21.15.131:33726) 2024-04-28T12:37:37.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.327+0000 7f2355800640 1 -- 172.21.15.131:0/1936027705 learned_addr learned my addr 172.21.15.131:0/1936027705 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:37.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.327+0000 7f234fe00640 1 -- 172.21.15.131:0/1936027705 >> v1:172.21.15.157:6789/0 conn(0x7f235015f180 legacy=0x7f23500d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:37.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.327+0000 7f234f400640 1 -- 172.21.15.131:0/1936027705 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3521425773 0 0) 0x7f23500d2d70 con 0x7f23500b3cb0 2024-04-28T12:37:37.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.327+0000 7f234fe00640 1 -- 172.21.15.131:0/1936027705 >> v1:172.21.15.131:6789/0 conn(0x7f23500b3cb0 legacy=0x7f23500b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.327+0000 7f234fe00640 1 -- 172.21.15.131:0/1936027705 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23500d09c0 con 0x7f23500d3830 2024-04-28T12:37:37.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.328+0000 7f234fe00640 1 --2- 172.21.15.131:0/1936027705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23500d3830 0x7f23500d56f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f233c0097a0 tx=0x7f233c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0100c32fae51610 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:37.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.329+0000 7f234f400640 1 -- 172.21.15.131:0/1936027705 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c037070 con 0x7f23500d3830 2024-04-28T12:37:37.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.329+0000 7f234f400640 1 -- 172.21.15.131:0/1936027705 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f233c032440 con 0x7f23500d3830 2024-04-28T12:37:37.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.329+0000 7f234f400640 1 -- 172.21.15.131:0/1936027705 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c032810 con 0x7f23500d3830 2024-04-28T12:37:37.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.329+0000 7f2356200640 1 -- 172.21.15.131:0/1936027705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23500d3830 msgr2=0x7f23500d56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.329+0000 7f2356200640 1 --2- 172.21.15.131:0/1936027705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23500d3830 0x7f23500d56f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f233c0097a0 tx=0x7f233c02a230 comp rx=0 tx=0).stop 2024-04-28T12:37:37.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.329+0000 7f2356200640 1 -- 172.21.15.131:0/1936027705 shutdown_connections 2024-04-28T12:37:37.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.329+0000 7f2356200640 1 --2- 172.21.15.131:0/1936027705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23500d3830 0x7f23500d56f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.329+0000 7f2356200640 1 -- 172.21.15.131:0/1936027705 >> 172.21.15.131:0/1936027705 conn(0x7f23500d4bd0 msgr2=0x7f23500d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:37.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.329+0000 7f2356200640 1 -- 172.21.15.131:0/1936027705 shutdown_connections 2024-04-28T12:37:37.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.329+0000 7f2356200640 1 -- 172.21.15.131:0/1936027705 wait complete. 2024-04-28T12:37:37.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.331+0000 7f2356200640 1 Processor -- start 2024-04-28T12:37:37.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.331+0000 7f2356200640 1 -- start start 2024-04-28T12:37:37.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.332+0000 7f2356200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23500b3cb0 0x7f23500cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:37.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.332+0000 7f2356200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23500d3830 0x7f2350158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:37.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.332+0000 7f2356200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f235015f180 0x7f23500b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:37.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.332+0000 7f2356200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f23500d11c0 con 0x7f23500b3cb0 2024-04-28T12:37:37.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.332+0000 7f2356200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f23500d2a40 con 0x7f235015f180 2024-04-28T12:37:37.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.332+0000 7f2356200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f23501f6630 con 0x7f23500d3830 2024-04-28T12:37:37.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.333+0000 7f2354e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23500b3cb0 0x7f23500cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:37.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.333+0000 7f2354e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23500b3cb0 0x7f23500cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47190/0 (socket says 172.21.15.131:47190) 2024-04-28T12:37:37.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.333+0000 7f2354e00640 1 -- 172.21.15.131:0/652459428 learned_addr learned my addr 172.21.15.131:0/652459428 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:37.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.333+0000 7f2354e00640 1 -- 172.21.15.131:0/652459428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23500d3830 msgr2=0x7f2350158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.333+0000 7f2354e00640 1 --2- 172.21.15.131:0/652459428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23500d3830 0x7f2350158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.333+0000 7f2354e00640 1 -- 172.21.15.131:0/652459428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f235015f180 msgr2=0x7f23500b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.333+0000 7f2354e00640 1 --2- 172.21.15.131:0/652459428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f235015f180 0x7f23500b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.333+0000 7f2354e00640 1 -- 172.21.15.131:0/652459428 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2350159bb0 con 0x7f23500b3cb0 2024-04-28T12:37:37.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.333+0000 7f2354e00640 1 --2- 172.21.15.131:0/652459428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23500b3cb0 0x7f23500cb110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f234000c730 tx=0x7f234000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:37.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.333+0000 7f234cc00640 1 -- 172.21.15.131:0/652459428 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340003b70 con 0x7f23500b3cb0 2024-04-28T12:37:37.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.333+0000 7f234cc00640 1 -- 172.21.15.131:0/652459428 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2340003d30 con 0x7f23500b3cb0 2024-04-28T12:37:37.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.334+0000 7f234cc00640 1 -- 172.21.15.131:0/652459428 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234000aa90 con 0x7f23500b3cb0 2024-04-28T12:37:37.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.334+0000 7f2356200640 1 -- 172.21.15.131:0/652459428 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23501595f0 con 0x7f23500b3cb0 2024-04-28T12:37:37.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.334+0000 7f2356200640 1 -- 172.21.15.131:0/652459428 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23501f62c0 con 0x7f23500b3cb0 2024-04-28T12:37:37.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.335+0000 7f2356200640 1 -- 172.21.15.131:0/652459428 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f235006f160 con 0x7f23500b3cb0 2024-04-28T12:37:37.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.336+0000 7f234cc00640 1 -- 172.21.15.131:0/652459428 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f234000ac30 con 0x7f23500b3cb0 2024-04-28T12:37:37.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.337+0000 7f234cc00640 1 --2- 172.21.15.131:0/652459428 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f233006c8f0 0x7f233006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:37.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.337+0000 7f234cc00640 1 -- 172.21.15.131:0/652459428 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f234008bdd0 con 0x7f23500b3cb0 2024-04-28T12:37:37.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.338+0000 7f234fe00640 1 --2- 172.21.15.131:0/652459428 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f233006c8f0 0x7f233006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:37.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.339+0000 7f234fe00640 1 --2- 172.21.15.131:0/652459428 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f233006c8f0 0x7f233006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f233c034000 tx=0x7f233c02c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:37.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.342+0000 7f234cc00640 1 -- 172.21.15.131:0/652459428 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f235006f160 con 0x7f23500b3cb0 2024-04-28T12:37:37.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.544+0000 7f2356200640 1 -- 172.21.15.131:0/652459428 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f23500c84b0 con 0x7f23500b3cb0 2024-04-28T12:37:37.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.546+0000 7f234cc00640 1 -- 172.21.15.131:0/652459428 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f23500c84b0 con 0x7f23500b3cb0 2024-04-28T12:37:37.543 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:37:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.548+0000 7f2356200640 1 -- 172.21.15.131:0/652459428 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f233006c8f0 msgr2=0x7f233006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.548+0000 7f2356200640 1 --2- 172.21.15.131:0/652459428 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f233006c8f0 0x7f233006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f233c034000 tx=0x7f233c02c000 comp rx=0 tx=0).stop 2024-04-28T12:37:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.548+0000 7f2356200640 1 -- 172.21.15.131:0/652459428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23500b3cb0 msgr2=0x7f23500cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.548+0000 7f2356200640 1 --2- 172.21.15.131:0/652459428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23500b3cb0 0x7f23500cb110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f234000c730 tx=0x7f234000cbf0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.548+0000 7f2356200640 1 -- 172.21.15.131:0/652459428 shutdown_connections 2024-04-28T12:37:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.548+0000 7f2356200640 1 --2- 172.21.15.131:0/652459428 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f233006c8f0 0x7f233006edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.548+0000 7f2356200640 1 --2- 172.21.15.131:0/652459428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f235015f180 0x7f23500b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.548+0000 7f2356200640 1 --2- 172.21.15.131:0/652459428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f23500d3830 0x7f2350158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.548+0000 7f2356200640 1 --2- 172.21.15.131:0/652459428 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23500b3cb0 0x7f23500cb110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.548+0000 7f2356200640 1 -- 172.21.15.131:0/652459428 >> 172.21.15.131:0/652459428 conn(0x7f23500d4bd0 msgr2=0x7f23500d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:37.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.549+0000 7f2356200640 1 -- 172.21.15.131:0/652459428 shutdown_connections 2024-04-28T12:37:37.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.549+0000 7f2356200640 1 -- 172.21.15.131:0/652459428 wait complete. 2024-04-28T12:37:37.564 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:37:37.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.765+0000 7ff3f9400640 1 Processor -- start 2024-04-28T12:37:37.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.765+0000 7ff3f9400640 1 -- start start 2024-04-28T12:37:37.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.767+0000 7ff3f9400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3f4145d80 0x7ff3f4072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:37.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.767+0000 7ff3f9400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3f415d5e0 0x7ff3f40728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:37.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.767+0000 7ff3f9400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3f4146160 con 0x7ff3f41468c0 2024-04-28T12:37:37.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.767+0000 7ff3f9400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff3f406dfb0 con 0x7ff3f415d5e0 2024-04-28T12:37:37.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.767+0000 7ff3f9400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f40d0470 con 0x7ff3f4145d80 2024-04-28T12:37:37.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.767+0000 7ff3f1e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3f415d5e0 0x7ff3f40728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:37.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.767+0000 7ff3f1e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3f415d5e0 0x7ff3f40728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47192/0 (socket says 172.21.15.131:47192) 2024-04-28T12:37:37.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.767+0000 7ff3f1e00640 1 -- 172.21.15.131:0/3801435839 learned_addr learned my addr 172.21.15.131:0/3801435839 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:37.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.767+0000 7ff3f2800640 1 --2- 172.21.15.131:0/3801435839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3f4145d80 0x7ff3f4072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:37.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.768+0000 7ff3f1e00640 1 -- 172.21.15.131:0/3801435839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3f4145d80 msgr2=0x7ff3f4072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:37.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.768+0000 7ff3f1e00640 1 --2- 172.21.15.131:0/3801435839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3f4145d80 0x7ff3f4072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.768+0000 7ff3f1e00640 1 -- 172.21.15.131:0/3801435839 >> v1:172.21.15.157:6789/0 conn(0x7ff3f41468c0 legacy=0x7ff3f406ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.768+0000 7ff3f1e00640 1 -- 172.21.15.131:0/3801435839 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f4074180 con 0x7ff3f415d5e0 2024-04-28T12:37:37.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.769+0000 7ff3f1e00640 1 --2- 172.21.15.131:0/3801435839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3f415d5e0 0x7ff3f40728c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009870 tx=0x7ff3dc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dc08aaec0c128e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:37.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.769+0000 7ff3f1400640 1 -- 172.21.15.131:0/3801435839 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc037070 con 0x7ff3f415d5e0 2024-04-28T12:37:37.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.769+0000 7ff3f1400640 1 -- 172.21.15.131:0/3801435839 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3dc032440 con 0x7ff3f415d5e0 2024-04-28T12:37:37.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.769+0000 7ff3f1400640 1 -- 172.21.15.131:0/3801435839 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc032810 con 0x7ff3f415d5e0 2024-04-28T12:37:37.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.770+0000 7ff3f9400640 1 -- 172.21.15.131:0/3801435839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3f415d5e0 msgr2=0x7ff3f40728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.770+0000 7ff3f9400640 1 --2- 172.21.15.131:0/3801435839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3f415d5e0 0x7ff3f40728c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009870 tx=0x7ff3dc02a1c0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.770+0000 7ff3f9400640 1 -- 172.21.15.131:0/3801435839 shutdown_connections 2024-04-28T12:37:37.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.770+0000 7ff3f9400640 1 --2- 172.21.15.131:0/3801435839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3f415d5e0 0x7ff3f40728c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.770+0000 7ff3f9400640 1 --2- 172.21.15.131:0/3801435839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3f4145d80 0x7ff3f4072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.770+0000 7ff3f9400640 1 -- 172.21.15.131:0/3801435839 >> 172.21.15.131:0/3801435839 conn(0x7ff3f40b2900 msgr2=0x7ff3f40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:37.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.770+0000 7ff3f9400640 1 -- 172.21.15.131:0/3801435839 shutdown_connections 2024-04-28T12:37:37.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.770+0000 7ff3f9400640 1 -- 172.21.15.131:0/3801435839 wait complete. 2024-04-28T12:37:37.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.772+0000 7ff3f9400640 1 Processor -- start 2024-04-28T12:37:37.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.772+0000 7ff3f9400640 1 -- start start 2024-04-28T12:37:37.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.773+0000 7ff3f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3f4145d80 0x7ff3f41f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:37.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.773+0000 7ff3f9400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3f41468c0 0x7ff3f413dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:37.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.773+0000 7ff3f9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3f415d5e0 0x7ff3f4141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:37.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.773+0000 7ff3f9400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f406e530 con 0x7ff3f4145d80 2024-04-28T12:37:37.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.773+0000 7ff3f9400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f406e250 con 0x7ff3f415d5e0 2024-04-28T12:37:37.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.773+0000 7ff3f9400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f40d09f0 con 0x7ff3f41468c0 2024-04-28T12:37:37.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.773+0000 7ff3f2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3f4145d80 0x7ff3f41f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:37.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.773+0000 7ff3f2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3f4145d80 0x7ff3f41f4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47204/0 (socket says 172.21.15.131:47204) 2024-04-28T12:37:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.773+0000 7ff3f2800640 1 -- 172.21.15.131:0/3001558355 learned_addr learned my addr 172.21.15.131:0/3001558355 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.773+0000 7ff3f1e00640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3f41468c0 0x7ff3f413dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.773+0000 7ff3f3200640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3f415d5e0 0x7ff3f4141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.774+0000 7ff3f2800640 1 -- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3f41468c0 msgr2=0x7ff3f413dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.774+0000 7ff3f2800640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3f41468c0 0x7ff3f413dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.774+0000 7ff3f2800640 1 -- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3f415d5e0 msgr2=0x7ff3f4141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.774+0000 7ff3f2800640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3f415d5e0 0x7ff3f4141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.774+0000 7ff3f2800640 1 -- 172.21.15.131:0/3001558355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f4155750 con 0x7ff3f4145d80 2024-04-28T12:37:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.774+0000 7ff3f2800640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3f4145d80 0x7ff3f41f4d10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4007e40 tx=0x7ff3e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.774+0000 7ff3eaa00640 1 -- 172.21.15.131:0/3001558355 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4014620 con 0x7ff3f4145d80 2024-04-28T12:37:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.774+0000 7ff3eaa00640 1 -- 172.21.15.131:0/3001558355 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e4019070 con 0x7ff3f4145d80 2024-04-28T12:37:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.774+0000 7ff3eaa00640 1 -- 172.21.15.131:0/3001558355 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4014a00 con 0x7ff3f4145d80 2024-04-28T12:37:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.774+0000 7ff3f9400640 1 -- 172.21.15.131:0/3001558355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f41559f0 con 0x7ff3f4145d80 2024-04-28T12:37:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.774+0000 7ff3f9400640 1 -- 172.21.15.131:0/3001558355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f4155fb0 con 0x7ff3f4145d80 2024-04-28T12:37:37.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.775+0000 7ff3f9400640 1 -- 172.21.15.131:0/3001558355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f4076260 con 0x7ff3f4145d80 2024-04-28T12:37:37.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.777+0000 7ff3eaa00640 1 -- 172.21.15.131:0/3001558355 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3e4014ba0 con 0x7ff3f4145d80 2024-04-28T12:37:37.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.777+0000 7ff3eaa00640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff3d006c8f0 0x7ff3d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:37.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.778+0000 7ff3eaa00640 1 -- 172.21.15.131:0/3001558355 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff3e408c7b0 con 0x7ff3f4145d80 2024-04-28T12:37:37.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.778+0000 7ff3f1e00640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff3d006c8f0 0x7ff3d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:37.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.779+0000 7ff3f1e00640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff3d006c8f0 0x7ff3d006edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc034000 tx=0x7ff3dc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.782+0000 7ff3eaa00640 1 -- 172.21.15.131:0/3001558355 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3f4076260 con 0x7ff3f4145d80 2024-04-28T12:37:37.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.984+0000 7ff3f9400640 1 -- 172.21.15.131:0/3001558355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff3f4069350 con 0x7ff3f4145d80 2024-04-28T12:37:37.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.985+0000 7ff3eaa00640 1 -- 172.21.15.131:0/3001558355 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7ff3e405a800 con 0x7ff3f4145d80 2024-04-28T12:37:37.982 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:37:37.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.989+0000 7ff3f9400640 1 -- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff3d006c8f0 msgr2=0x7ff3d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.989+0000 7ff3f9400640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff3d006c8f0 0x7ff3d006edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc034000 tx=0x7ff3dc02e000 comp rx=0 tx=0).stop 2024-04-28T12:37:37.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.989+0000 7ff3f9400640 1 -- 172.21.15.131:0/3001558355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3f4145d80 msgr2=0x7ff3f41f4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:37.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.989+0000 7ff3f9400640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3f4145d80 0x7ff3f41f4d10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4007e40 tx=0x7ff3e400e4b0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.989+0000 7ff3f9400640 1 -- 172.21.15.131:0/3001558355 shutdown_connections 2024-04-28T12:37:37.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.989+0000 7ff3f9400640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff3d006c8f0 0x7ff3d006edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.989+0000 7ff3f9400640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3f415d5e0 0x7ff3f4141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.989+0000 7ff3f9400640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3f41468c0 0x7ff3f413dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.989+0000 7ff3f9400640 1 --2- 172.21.15.131:0/3001558355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3f4145d80 0x7ff3f41f4d10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:37.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.989+0000 7ff3f9400640 1 -- 172.21.15.131:0/3001558355 >> 172.21.15.131:0/3001558355 conn(0x7ff3f40b2900 msgr2=0x7ff3f4144f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:37.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.989+0000 7ff3f9400640 1 -- 172.21.15.131:0/3001558355 shutdown_connections 2024-04-28T12:37:37.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:37.989+0000 7ff3f9400640 1 -- 172.21.15.131:0/3001558355 wait complete. 2024-04-28T12:37:38.005 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:37:38.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.214+0000 7fa66fc00640 1 Processor -- start 2024-04-28T12:37:38.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.215+0000 7fa66fc00640 1 -- start start 2024-04-28T12:37:38.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.216+0000 7fa66fc00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa6680b4f40 0x7fa66806e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.216+0000 7fa66fc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6680b3c00 con 0x7fa668152720 2024-04-28T12:37:38.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.216+0000 7fa66fc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa668152b00 con 0x7fa66806ed10 2024-04-28T12:37:38.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.216+0000 7fa66fc00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa66806f140 con 0x7fa6680b4f40 2024-04-28T12:37:38.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.216+0000 7fa66d200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa668152720 0x7fa66815c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33732/0 (socket says 172.21.15.131:33732) 2024-04-28T12:37:38.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.216+0000 7fa66d200640 1 -- 172.21.15.131:0/1010775335 learned_addr learned my addr 172.21.15.131:0/1010775335 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:38.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa65fe00640 1 --2- 172.21.15.131:0/1010775335 >> v2:172.21.15.157:3300/0 conn(0x7fa6680b4f40 0x7fa66806e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2673453724 0 0) 0x7fa6680b3c00 con 0x7fa668152720 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa64c003440 con 0x7fa668152720 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1442604831 0 0) 0x7fa64c003440 con 0x7fa668152720 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 >> v2:172.21.15.157:3300/0 conn(0x7fa6680b4f40 msgr2=0x7fa66806e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 --2- 172.21.15.131:0/1010775335 >> v2:172.21.15.157:3300/0 conn(0x7fa6680b4f40 0x7fa66806e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 >> v1:172.21.15.157:6789/0 conn(0x7fa66806ed10 legacy=0x7fa66815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6680b3ea0 con 0x7fa668152720 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fa658001670 con 0x7fa668152720 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 >> v1:172.21.15.131:6789/0 conn(0x7fa668152720 legacy=0x7fa66815c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 0x7fa64c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa64c0088a0 0x7fa64c00ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa64c00b510 0x7fa64c00d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa66806f140 con 0x7fa64c003af0 2024-04-28T12:37:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c00e2b0 con 0x7fa64c00b510 2024-04-28T12:37:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.217+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa64c00e4b0 con 0x7fa64c0088a0 2024-04-28T12:37:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.218+0000 7fa66d200640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 0x7fa64c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.218+0000 7fa65fe00640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa64c0088a0 0x7fa64c00ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.218+0000 7fa66dc00640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa64c00b510 0x7fa64c00d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.218+0000 7fa66d200640 1 -- 172.21.15.131:0/1010775335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa64c0088a0 msgr2=0x7fa64c00ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.218+0000 7fa66d200640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa64c0088a0 0x7fa64c00ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.218+0000 7fa66d200640 1 -- 172.21.15.131:0/1010775335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa64c00b510 msgr2=0x7fa64c00d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.218+0000 7fa66d200640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa64c00b510 0x7fa64c00d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.218+0000 7fa66d200640 1 -- 172.21.15.131:0/1010775335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c003440 con 0x7fa64c003af0 2024-04-28T12:37:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.218+0000 7fa65fe00640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa64c0088a0 0x7fa64c00ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.218+0000 7fa66dc00640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa64c00b510 0x7fa64c00d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.218+0000 7fa66dc00640 1 -- 172.21.15.131:0/1010775335 reap_dead start 2024-04-28T12:37:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.219+0000 7fa66d200640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 0x7fa64c003f10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c110 tx=0x7fa64c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39797fd8498b7c56 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.219+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658007c40 con 0x7fa64c003af0 2024-04-28T12:37:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.219+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa658005af0 con 0x7fa64c003af0 2024-04-28T12:37:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.219+0000 7fa667e00640 1 -- 172.21.15.131:0/1010775335 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658007070 con 0x7fa64c003af0 2024-04-28T12:37:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.220+0000 7fa66fc00640 1 -- 172.21.15.131:0/1010775335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 msgr2=0x7fa64c003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.220+0000 7fa66fc00640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 0x7fa64c003f10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c110 tx=0x7fa64c003730 comp rx=0 tx=0).stop 2024-04-28T12:37:38.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.220+0000 7fa66fc00640 1 -- 172.21.15.131:0/1010775335 shutdown_connections 2024-04-28T12:37:38.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.220+0000 7fa66fc00640 1 --2- 172.21.15.131:0/1010775335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 0x7fa64c003f10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.220+0000 7fa66fc00640 1 -- 172.21.15.131:0/1010775335 >> 172.21.15.131:0/1010775335 conn(0x7fa6680b2900 msgr2=0x7fa6680b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:38.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.220+0000 7fa66fc00640 1 -- 172.21.15.131:0/1010775335 shutdown_connections 2024-04-28T12:37:38.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.220+0000 7fa66fc00640 1 -- 172.21.15.131:0/1010775335 wait complete. 2024-04-28T12:37:38.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.222+0000 7fa66fc00640 1 Processor -- start 2024-04-28T12:37:38.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.222+0000 7fa66fc00640 1 -- start start 2024-04-28T12:37:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.223+0000 7fa66fc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 0x7fa6681587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.223+0000 7fa66fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa66814b810 0x7fa668147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.223+0000 7fa66fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa66814c320 0x7fa66814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.223+0000 7fa66fc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6680b4740 con 0x7fa64c003af0 2024-04-28T12:37:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.223+0000 7fa66fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6680b4460 con 0x7fa66814c320 2024-04-28T12:37:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.223+0000 7fa66fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa6680b4a20 con 0x7fa66814b810 2024-04-28T12:37:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.223+0000 7fa66d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 0x7fa6681587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.223+0000 7fa66d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 0x7fa6681587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47228/0 (socket says 172.21.15.131:47228) 2024-04-28T12:37:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.223+0000 7fa66d200640 1 -- 172.21.15.131:0/4109988057 learned_addr learned my addr 172.21.15.131:0/4109988057 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.224+0000 7fa66dc00640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa66814c320 0x7fa66814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.224+0000 7fa65fe00640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa66814b810 0x7fa668147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.224+0000 7fa66d200640 1 -- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa66814b810 msgr2=0x7fa668147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.224+0000 7fa66d200640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa66814b810 0x7fa668147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.224+0000 7fa66d200640 1 -- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa66814c320 msgr2=0x7fa66814c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.224+0000 7fa66d200640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa66814c320 0x7fa66814c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.224+0000 7fa66d200640 1 -- 172.21.15.131:0/4109988057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa668159040 con 0x7fa64c003af0 2024-04-28T12:37:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.224+0000 7fa66d200640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 0x7fa6681587f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa65802ad60 tx=0x7fa66815bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.224+0000 7fa665600640 1 -- 172.21.15.131:0/4109988057 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658001670 con 0x7fa64c003af0 2024-04-28T12:37:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.224+0000 7fa665600640 1 -- 172.21.15.131:0/4109988057 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa658042070 con 0x7fa64c003af0 2024-04-28T12:37:38.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.224+0000 7fa665600640 1 -- 172.21.15.131:0/4109988057 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658007290 con 0x7fa64c003af0 2024-04-28T12:37:38.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.224+0000 7fa66fc00640 1 -- 172.21.15.131:0/4109988057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa668159320 con 0x7fa64c003af0 2024-04-28T12:37:38.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.225+0000 7fa66fc00640 1 -- 172.21.15.131:0/4109988057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa668072760 con 0x7fa64c003af0 2024-04-28T12:37:38.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.226+0000 7fa66fc00640 1 -- 172.21.15.131:0/4109988057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa668069350 con 0x7fa64c003af0 2024-04-28T12:37:38.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.227+0000 7fa665600640 1 -- 172.21.15.131:0/4109988057 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa658007430 con 0x7fa64c003af0 2024-04-28T12:37:38.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.228+0000 7fa665600640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa64406c920 0x7fa64406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.228+0000 7fa665600640 1 -- 172.21.15.131:0/4109988057 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa6580abff0 con 0x7fa64c003af0 2024-04-28T12:37:38.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.228+0000 7fa65fe00640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa64406c920 0x7fa64406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.229+0000 7fa65fe00640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa64406c920 0x7fa64406ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa6680746a0 tx=0x7fa650008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:38.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.232+0000 7fa665600640 1 -- 172.21.15.131:0/4109988057 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa65807a0d0 con 0x7fa64c003af0 2024-04-28T12:37:38.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.429+0000 7fa66fc00640 1 -- 172.21.15.131:0/4109988057 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa6680d0040 con 0x7fa64c003af0 2024-04-28T12:37:38.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.429+0000 7fa665600640 1 -- 172.21.15.131:0/4109988057 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fa6580799d0 con 0x7fa64c003af0 2024-04-28T12:37:38.426 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:37:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.432+0000 7fa66fc00640 1 -- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa64406c920 msgr2=0x7fa64406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.432+0000 7fa66fc00640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa64406c920 0x7fa64406ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa6680746a0 tx=0x7fa650008040 comp rx=0 tx=0).stop 2024-04-28T12:37:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.432+0000 7fa66fc00640 1 -- 172.21.15.131:0/4109988057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 msgr2=0x7fa6681587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.432+0000 7fa66fc00640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 0x7fa6681587f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa65802ad60 tx=0x7fa66815bc00 comp rx=0 tx=0).stop 2024-04-28T12:37:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.432+0000 7fa66fc00640 1 -- 172.21.15.131:0/4109988057 shutdown_connections 2024-04-28T12:37:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.432+0000 7fa66fc00640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa64406c920 0x7fa64406ede0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.432+0000 7fa66fc00640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa66814c320 0x7fa66814c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.432+0000 7fa66fc00640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa66814b810 0x7fa668147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.432+0000 7fa66fc00640 1 --2- 172.21.15.131:0/4109988057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa64c003af0 0x7fa6681587f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.432+0000 7fa66fc00640 1 -- 172.21.15.131:0/4109988057 >> 172.21.15.131:0/4109988057 conn(0x7fa6680b2900 msgr2=0x7fa66815a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:38.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.432+0000 7fa66fc00640 1 -- 172.21.15.131:0/4109988057 shutdown_connections 2024-04-28T12:37:38.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.432+0000 7fa66fc00640 1 -- 172.21.15.131:0/4109988057 wait complete. 2024-04-28T12:37:38.449 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T12:37:38.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.655+0000 7fcd74c00640 1 Processor -- start 2024-04-28T12:37:38.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.656+0000 7fcd74c00640 1 -- start start 2024-04-28T12:37:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.657+0000 7fcd74c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd700afe90 0x7fcd700b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.657+0000 7fcd74c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcd700d1570 0x7fcd700d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.657+0000 7fcd74c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd7015f360 con 0x7fcd700b6ae0 2024-04-28T12:37:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.657+0000 7fcd74c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcd700b0ad0 con 0x7fcd700d1570 2024-04-28T12:37:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.657+0000 7fcd74c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd700b0db0 con 0x7fcd700afe90 2024-04-28T12:37:38.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.658+0000 7fcd6ea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcd700b6ae0 0x7fcd7015faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33746/0 (socket says 172.21.15.131:33746) 2024-04-28T12:37:38.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.658+0000 7fcd6ea00640 1 -- 172.21.15.131:0/3059971130 learned_addr learned my addr 172.21.15.131:0/3059971130 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.658+0000 7fcd6fe00640 1 --2- 172.21.15.131:0/3059971130 >> v2:172.21.15.157:3300/0 conn(0x7fcd700d1570 0x7fcd700d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.658+0000 7fcd6f400640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd700afe90 0x7fcd700b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.658+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2621679043 0 0) 0x7fcd7015f360 con 0x7fcd700b6ae0 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.658+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd54003520 con 0x7fcd700b6ae0 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1248310263 0 0) 0x7fcd54003520 con 0x7fcd700b6ae0 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd700afe90 msgr2=0x7fcd700b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd700afe90 0x7fcd700b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 >> v2:172.21.15.157:3300/0 conn(0x7fcd700d1570 msgr2=0x7fcd700d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 --2- 172.21.15.131:0/3059971130 >> v2:172.21.15.157:3300/0 conn(0x7fcd700d1570 0x7fcd700d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd700b1090 con 0x7fcd700b6ae0 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fcd58001670 con 0x7fcd700b6ae0 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 >> v1:172.21.15.131:6789/0 conn(0x7fcd700b6ae0 legacy=0x7fcd7015faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd540039b0 0x7fcd54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd54004800 0x7fcd54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd700afe90 0x7fcd540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcd700b0ad0 con 0x7fcd54004800 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd700b0db0 con 0x7fcd540039b0 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5400b9c0 con 0x7fcd700afe90 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6ea00640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd54004800 0x7fcd54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6f400640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd540039b0 0x7fcd54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6fe00640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd700afe90 0x7fcd540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6ea00640 1 -- 172.21.15.131:0/3059971130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd700afe90 msgr2=0x7fcd540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6ea00640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd700afe90 0x7fcd540051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6ea00640 1 -- 172.21.15.131:0/3059971130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd540039b0 msgr2=0x7fcd54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6ea00640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd540039b0 0x7fcd54003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.659+0000 7fcd6ea00640 1 -- 172.21.15.131:0/3059971130 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd54003520 con 0x7fcd54004800 2024-04-28T12:37:38.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.660+0000 7fcd6f400640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd540039b0 0x7fcd54003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.661+0000 7fcd6ea00640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd54004800 0x7fcd54004c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcd700b6ec0 tx=0x7fcd54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75955c906b8c7420 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.661+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58007d50 con 0x7fcd54004800 2024-04-28T12:37:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.661+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd580058d0 con 0x7fcd54004800 2024-04-28T12:37:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.661+0000 7fcd6e000640 1 -- 172.21.15.131:0/3059971130 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58005cc0 con 0x7fcd54004800 2024-04-28T12:37:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.661+0000 7fcd74c00640 1 -- 172.21.15.131:0/3059971130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd54004800 msgr2=0x7fcd54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.661+0000 7fcd74c00640 1 --2- 172.21.15.131:0/3059971130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd54004800 0x7fcd54004c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcd700b6ec0 tx=0x7fcd54005830 comp rx=0 tx=0).stop 2024-04-28T12:37:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.661+0000 7fcd6fe00640 1 -- 172.21.15.131:0/3059971130 reap_dead start 2024-04-28T12:37:38.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.662+0000 7fcd74c00640 1 -- 172.21.15.131:0/3059971130 shutdown_connections 2024-04-28T12:37:38.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.662+0000 7fcd74c00640 1 -- 172.21.15.131:0/3059971130 >> 172.21.15.131:0/3059971130 conn(0x7fcd70075df0 msgr2=0x7fcd70068090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:38.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.662+0000 7fcd74c00640 1 -- 172.21.15.131:0/3059971130 shutdown_connections 2024-04-28T12:37:38.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.662+0000 7fcd74c00640 1 -- 172.21.15.131:0/3059971130 wait complete. 2024-04-28T12:37:38.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.663+0000 7fcd74c00640 1 Processor -- start 2024-04-28T12:37:38.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.664+0000 7fcd74c00640 1 -- start start 2024-04-28T12:37:38.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.664+0000 7fcd74c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd54004800 0x7fcd70146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.664+0000 7fcd74c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd70154430 0x7fcd7015b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.664+0000 7fcd74c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd7015bb60 0x7fcd700b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd74c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcd700b0760 con 0x7fcd54004800 2024-04-28T12:37:38.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd74c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70070790 con 0x7fcd70154430 2024-04-28T12:37:38.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd74c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd700b0270 con 0x7fcd7015bb60 2024-04-28T12:37:38.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd6f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd54004800 0x7fcd70146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd6f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd54004800 0x7fcd70146320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47236/0 (socket says 172.21.15.131:47236) 2024-04-28T12:37:38.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd6f400640 1 -- 172.21.15.131:0/1173186065 learned_addr learned my addr 172.21.15.131:0/1173186065 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:38.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd6ea00640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd70154430 0x7fcd7015b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd6f400640 1 -- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd7015bb60 msgr2=0x7fcd700b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd6fe00640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd7015bb60 0x7fcd700b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd6f400640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd7015bb60 0x7fcd700b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd6f400640 1 -- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd70154430 msgr2=0x7fcd7015b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd6f400640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd70154430 0x7fcd7015b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd6f400640 1 -- 172.21.15.131:0/1173186065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd70146860 con 0x7fcd54004800 2024-04-28T12:37:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.665+0000 7fcd6ea00640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd70154430 0x7fcd7015b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.666+0000 7fcd6f400640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd54004800 0x7fcd70146320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000f350 tx=0x7fcd6000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.666+0000 7fcd67400640 1 -- 172.21.15.131:0/1173186065 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60003b90 con 0x7fcd54004800 2024-04-28T12:37:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.666+0000 7fcd67400640 1 -- 172.21.15.131:0/1173186065 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd60003d30 con 0x7fcd54004800 2024-04-28T12:37:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.666+0000 7fcd67400640 1 -- 172.21.15.131:0/1173186065 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60004240 con 0x7fcd54004800 2024-04-28T12:37:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.666+0000 7fcd74c00640 1 -- 172.21.15.131:0/1173186065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd70153390 con 0x7fcd54004800 2024-04-28T12:37:38.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.666+0000 7fcd74c00640 1 -- 172.21.15.131:0/1173186065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd70153950 con 0x7fcd54004800 2024-04-28T12:37:38.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.667+0000 7fcd74c00640 1 -- 172.21.15.131:0/1173186065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7015bf70 con 0x7fcd54004800 2024-04-28T12:37:38.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.668+0000 7fcd67400640 1 -- 172.21.15.131:0/1173186065 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd600043e0 con 0x7fcd54004800 2024-04-28T12:37:38.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.669+0000 7fcd67400640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcd4c06c920 0x7fcd4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:38.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.669+0000 7fcd67400640 1 -- 172.21.15.131:0/1173186065 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcd6008c3c0 con 0x7fcd54004800 2024-04-28T12:37:38.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.670+0000 7fcd6ea00640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcd4c06c920 0x7fcd4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:38.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.671+0000 7fcd6ea00640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcd4c06c920 0x7fcd4c06ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd58033000 tx=0x7fcd58009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:38.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.673+0000 7fcd67400640 1 -- 172.21.15.131:0/1173186065 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd7015bf70 con 0x7fcd54004800 2024-04-28T12:37:38.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.897+0000 7fcd74c00640 1 -- 172.21.15.131:0/1173186065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcd70069a60 con 0x7fcd54004800 2024-04-28T12:37:38.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.898+0000 7fcd67400640 1 -- 172.21.15.131:0/1173186065 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fcd70146860 con 0x7fcd54004800 2024-04-28T12:37:38.895 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-04-28T12:37:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.901+0000 7fcd74c00640 1 -- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcd4c06c920 msgr2=0x7fcd4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.901+0000 7fcd74c00640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcd4c06c920 0x7fcd4c06ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd58033000 tx=0x7fcd58009030 comp rx=0 tx=0).stop 2024-04-28T12:37:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.901+0000 7fcd74c00640 1 -- 172.21.15.131:0/1173186065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd54004800 msgr2=0x7fcd70146320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.901+0000 7fcd74c00640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd54004800 0x7fcd70146320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000f350 tx=0x7fcd6000f810 comp rx=0 tx=0).stop 2024-04-28T12:37:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.901+0000 7fcd74c00640 1 -- 172.21.15.131:0/1173186065 shutdown_connections 2024-04-28T12:37:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.901+0000 7fcd74c00640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcd4c06c920 0x7fcd4c06ede0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.901+0000 7fcd74c00640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd7015bb60 0x7fcd700b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.901+0000 7fcd74c00640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd70154430 0x7fcd7015b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.901+0000 7fcd74c00640 1 --2- 172.21.15.131:0/1173186065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd54004800 0x7fcd70146320 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:38.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.901+0000 7fcd74c00640 1 -- 172.21.15.131:0/1173186065 >> 172.21.15.131:0/1173186065 conn(0x7fcd70075df0 msgr2=0x7fcd700b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:38.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.901+0000 7fcd74c00640 1 -- 172.21.15.131:0/1173186065 shutdown_connections 2024-04-28T12:37:38.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:38.901+0000 7fcd74c00640 1 -- 172.21.15.131:0/1173186065 wait complete. 2024-04-28T12:37:38.917 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:37:38.917 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:37:38.967 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:39.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.215+0000 7f8edb200640 1 Processor -- start 2024-04-28T12:37:39.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.215+0000 7f8edb200640 1 -- start start 2024-04-28T12:37:39.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.216+0000 7f8edb200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8edc0d1620 0x7f8edc0d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:39.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.217+0000 7f8edb200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8edc0d3810 0x7f8edc0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:39.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.217+0000 7f8edb200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8edc0d0480 con 0x7f8edc0d1a00 2024-04-28T12:37:39.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.217+0000 7f8edb200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f8edc0d0ce0 con 0x7f8edc0d3810 2024-04-28T12:37:39.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.217+0000 7f8edb200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8edc0d0720 con 0x7f8edc0d1620 2024-04-28T12:37:39.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.217+0000 7f8eda800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8edc0d3810 0x7f8edc0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.217+0000 7f8ed3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8edc0d1a00 0x7f8edc0b4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33762/0 (socket says 172.21.15.131:33762) 2024-04-28T12:37:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.217+0000 7f8ed3e00640 1 -- 172.21.15.131:0/2428276230 learned_addr learned my addr 172.21.15.131:0/2428276230 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.217+0000 7f8ed9e00640 1 --2- 172.21.15.131:0/2428276230 >> v2:172.21.15.157:3300/0 conn(0x7f8edc0d1620 0x7f8edc0d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.218+0000 7f8eda800640 1 -- 172.21.15.131:0/2428276230 >> v2:172.21.15.157:3300/0 conn(0x7f8edc0d1620 msgr2=0x7f8edc0d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.218+0000 7f8eda800640 1 --2- 172.21.15.131:0/2428276230 >> v2:172.21.15.157:3300/0 conn(0x7f8edc0d1620 0x7f8edc0d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.218+0000 7f8eda800640 1 -- 172.21.15.131:0/2428276230 >> v1:172.21.15.131:6789/0 conn(0x7f8edc0d1a00 legacy=0x7f8edc0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.218+0000 7f8eda800640 1 -- 172.21.15.131:0/2428276230 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc0d0a00 con 0x7f8edc0d3810 2024-04-28T12:37:39.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.218+0000 7f8ed9400640 1 -- 172.21.15.131:0/2428276230 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3658655263 0 0) 0x7f8edc0d0480 con 0x7f8edc0d1a00 2024-04-28T12:37:39.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.219+0000 7f8eda800640 1 --2- 172.21.15.131:0/2428276230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8edc0d3810 0x7f8edc0b5280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8edc08d330 tx=0x7f8ed402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317ecc62313df065 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:39.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.219+0000 7f8ed9400640 1 -- 172.21.15.131:0/2428276230 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4007740 con 0x7f8edc0d3810 2024-04-28T12:37:39.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.219+0000 7f8ed9400640 1 -- 172.21.15.131:0/2428276230 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed40078e0 con 0x7f8edc0d3810 2024-04-28T12:37:39.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.219+0000 7f8ed9400640 1 -- 172.21.15.131:0/2428276230 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4007cb0 con 0x7f8edc0d3810 2024-04-28T12:37:39.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.219+0000 7f8edb200640 1 -- 172.21.15.131:0/2428276230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8edc0d3810 msgr2=0x7f8edc0b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.219+0000 7f8edb200640 1 --2- 172.21.15.131:0/2428276230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8edc0d3810 0x7f8edc0b5280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8edc08d330 tx=0x7f8ed402a5f0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.220+0000 7f8edb200640 1 -- 172.21.15.131:0/2428276230 shutdown_connections 2024-04-28T12:37:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.220+0000 7f8edb200640 1 --2- 172.21.15.131:0/2428276230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8edc0d3810 0x7f8edc0b5280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.220+0000 7f8edb200640 1 --2- 172.21.15.131:0/2428276230 >> v2:172.21.15.157:3300/0 conn(0x7f8edc0d1620 0x7f8edc0d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.220+0000 7f8edb200640 1 -- 172.21.15.131:0/2428276230 >> 172.21.15.131:0/2428276230 conn(0x7f8edc0d4c30 msgr2=0x7f8edc0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.220+0000 7f8edb200640 1 -- 172.21.15.131:0/2428276230 shutdown_connections 2024-04-28T12:37:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.220+0000 7f8edb200640 1 -- 172.21.15.131:0/2428276230 wait complete. 2024-04-28T12:37:39.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.221+0000 7f8edb200640 1 Processor -- start 2024-04-28T12:37:39.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.222+0000 7f8edb200640 1 -- start start 2024-04-28T12:37:39.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.222+0000 7f8edb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc0d1620 0x7f8edc0ba4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:39.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8edb200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8edc0d1a00 0x7f8edc0b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:39.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8edb200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0d3810 0x7f8edc14b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:39.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8edb200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc0b57c0 con 0x7f8edc0d1a00 2024-04-28T12:37:39.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8edb200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc0d1e60 con 0x7f8edc0d3810 2024-04-28T12:37:39.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8edb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8edc0b5940 con 0x7f8edc0d1620 2024-04-28T12:37:39.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8edc0d1a00 0x7f8edc0b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:39.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8ed3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8edc0d1a00 0x7f8edc0b7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47260/0 (socket says 172.21.15.131:47260) 2024-04-28T12:37:39.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8ed3e00640 1 -- 172.21.15.131:0/3369865919 learned_addr learned my addr 172.21.15.131:0/3369865919 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:39.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8ed9e00640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc0d1620 0x7f8edc0ba4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:39.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8ed3e00640 1 -- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc0d1620 msgr2=0x7f8edc0ba4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8ed3e00640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc0d1620 0x7f8edc0ba4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8ed3e00640 1 -- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0d3810 msgr2=0x7f8edc14b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8ed3e00640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0d3810 0x7f8edc14b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8ed3e00640 1 -- 172.21.15.131:0/3369865919 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc1f5960 con 0x7f8edc0d1a00 2024-04-28T12:37:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.223+0000 7f8ed9e00640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc0d1620 0x7f8edc0ba4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.224+0000 7f8ed3e00640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8edc0d1a00 0x7f8edc0b7a20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400c4d0 tx=0x7f8ec400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.224+0000 7f8ed2000640 1 -- 172.21.15.131:0/3369865919 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4017070 con 0x7f8edc0d1a00 2024-04-28T12:37:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.224+0000 7f8ed2000640 1 -- 172.21.15.131:0/3369865919 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec4005d00 con 0x7f8edc0d1a00 2024-04-28T12:37:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.224+0000 7f8edb200640 1 -- 172.21.15.131:0/3369865919 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8edc1f5680 con 0x7f8edc0d1a00 2024-04-28T12:37:39.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.224+0000 7f8ed2000640 1 -- 172.21.15.131:0/3369865919 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40072b0 con 0x7f8edc0d1a00 2024-04-28T12:37:39.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.224+0000 7f8edb200640 1 -- 172.21.15.131:0/3369865919 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8edc0b9cb0 con 0x7f8edc0d1a00 2024-04-28T12:37:39.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.225+0000 7f8edb200640 1 -- 172.21.15.131:0/3369865919 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8edc0b97e0 con 0x7f8edc0d1a00 2024-04-28T12:37:39.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.226+0000 7f8ed2000640 1 -- 172.21.15.131:0/3369865919 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ec401a070 con 0x7f8edc0d1a00 2024-04-28T12:37:39.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.227+0000 7f8ed2000640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8eb806c8f0 0x7f8eb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:39.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.227+0000 7f8ed2000640 1 -- 172.21.15.131:0/3369865919 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8ec408b320 con 0x7f8edc0d1a00 2024-04-28T12:37:39.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.228+0000 7f8ed9e00640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8eb806c8f0 0x7f8eb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:39.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.229+0000 7f8ed9e00640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8eb806c8f0 0x7f8eb806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc006fd0 tx=0x7f8ecc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:39.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.232+0000 7f8ed2000640 1 -- 172.21.15.131:0/3369865919 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ec4059370 con 0x7f8edc0d1a00 2024-04-28T12:37:39.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.434+0000 7f8edb200640 1 -- 172.21.15.131:0/3369865919 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8edc069350 con 0x7f8edc0d1a00 2024-04-28T12:37:39.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.436+0000 7f8ed2000640 1 -- 172.21.15.131:0/3369865919 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7f8ec405f030 con 0x7f8edc0d1a00 2024-04-28T12:37:39.433 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:39.433 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:39.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.439+0000 7f8edb200640 1 -- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8eb806c8f0 msgr2=0x7f8eb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:39.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.439+0000 7f8edb200640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8eb806c8f0 0x7f8eb806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc006fd0 tx=0x7f8ecc008040 comp rx=0 tx=0).stop 2024-04-28T12:37:39.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.439+0000 7f8edb200640 1 -- 172.21.15.131:0/3369865919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8edc0d1a00 msgr2=0x7f8edc0b7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.439+0000 7f8edb200640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8edc0d1a00 0x7f8edc0b7a20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400c4d0 tx=0x7f8ec400c990 comp rx=0 tx=0).stop 2024-04-28T12:37:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.439+0000 7f8edb200640 1 -- 172.21.15.131:0/3369865919 shutdown_connections 2024-04-28T12:37:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.439+0000 7f8edb200640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8eb806c8f0 0x7f8eb806edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.439+0000 7f8edb200640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8edc0d3810 0x7f8edc14b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.439+0000 7f8edb200640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8edc0d1a00 0x7f8edc0b7a20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.439+0000 7f8edb200640 1 --2- 172.21.15.131:0/3369865919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8edc0d1620 0x7f8edc0ba4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.439+0000 7f8edb200640 1 -- 172.21.15.131:0/3369865919 >> 172.21.15.131:0/3369865919 conn(0x7f8edc0d4c30 msgr2=0x7f8edc06d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.440+0000 7f8edb200640 1 -- 172.21.15.131:0/3369865919 shutdown_connections 2024-04-28T12:37:39.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.440+0000 7f8edb200640 1 -- 172.21.15.131:0/3369865919 wait complete. 2024-04-28T12:37:39.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:37:39.455 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:37:39.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.662+0000 7fc7ff200640 1 Processor -- start 2024-04-28T12:37:39.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.662+0000 7fc7ff200640 1 -- start start 2024-04-28T12:37:39.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.663+0000 7fc7ff200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc7f80cf440 0x7fc7f80b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:39.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.663+0000 7fc7ff200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80cf820 0x7fc7f80cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:39.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.664+0000 7fc7ff200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f806eaf0 con 0x7fc7f80d01c0 2024-04-28T12:37:39.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.664+0000 7fc7ff200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f806f420 con 0x7fc7f80cf440 2024-04-28T12:37:39.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.664+0000 7fc7ff200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f806edd0 con 0x7fc7f80cf820 2024-04-28T12:37:39.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.664+0000 7fc7f7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc7f80cf440 0x7fc7f80b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:39.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.664+0000 7fc7fd200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc7f80d01c0 0x7fc7f80b41d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33770/0 (socket says 172.21.15.131:33770) 2024-04-28T12:37:39.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.664+0000 7fc7fd200640 1 -- 172.21.15.131:0/2214808348 learned_addr learned my addr 172.21.15.131:0/2214808348 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:39.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.664+0000 7fc7ef400640 1 --2- 172.21.15.131:0/2214808348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80cf820 0x7fc7f80cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:39.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.664+0000 7fc7f7e00640 1 -- 172.21.15.131:0/2214808348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80cf820 msgr2=0x7fc7f80cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:39.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.664+0000 7fc7f7e00640 1 --2- 172.21.15.131:0/2214808348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80cf820 0x7fc7f80cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.664+0000 7fc7f7e00640 1 -- 172.21.15.131:0/2214808348 >> v1:172.21.15.131:6789/0 conn(0x7fc7f80d01c0 legacy=0x7fc7f80b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:39.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.665+0000 7fc7f7400640 1 -- 172.21.15.131:0/2214808348 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1393034112 0 0) 0x7fc7f806eaf0 con 0x7fc7f80d01c0 2024-04-28T12:37:39.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.665+0000 7fc7f7e00640 1 -- 172.21.15.131:0/2214808348 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f806f0b0 con 0x7fc7f80cf440 2024-04-28T12:37:39.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.666+0000 7fc7f7e00640 1 --2- 172.21.15.131:0/2214808348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f80cf440 0x7fc7f80b5250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80097a0 tx=0x7fc7e802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3342a9f26e3cef7a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:39.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.666+0000 7fc7f7400640 1 -- 172.21.15.131:0/2214808348 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8037070 con 0x7fc7f80cf440 2024-04-28T12:37:39.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.666+0000 7fc7f7400640 1 -- 172.21.15.131:0/2214808348 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e802ae50 con 0x7fc7f80cf440 2024-04-28T12:37:39.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.666+0000 7fc7f7400640 1 -- 172.21.15.131:0/2214808348 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8032660 con 0x7fc7f80cf440 2024-04-28T12:37:39.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.666+0000 7fc7ff200640 1 -- 172.21.15.131:0/2214808348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f80cf440 msgr2=0x7fc7f80b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.666+0000 7fc7ff200640 1 --2- 172.21.15.131:0/2214808348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f80cf440 0x7fc7f80b5250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80097a0 tx=0x7fc7e802a100 comp rx=0 tx=0).stop 2024-04-28T12:37:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.667+0000 7fc7ff200640 1 -- 172.21.15.131:0/2214808348 shutdown_connections 2024-04-28T12:37:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.667+0000 7fc7ff200640 1 --2- 172.21.15.131:0/2214808348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80cf820 0x7fc7f80cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.667+0000 7fc7ff200640 1 --2- 172.21.15.131:0/2214808348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f80cf440 0x7fc7f80b5250 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.667+0000 7fc7ff200640 1 -- 172.21.15.131:0/2214808348 >> 172.21.15.131:0/2214808348 conn(0x7fc7f80b2900 msgr2=0x7fc7f80b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.667+0000 7fc7ff200640 1 -- 172.21.15.131:0/2214808348 shutdown_connections 2024-04-28T12:37:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.667+0000 7fc7ff200640 1 -- 172.21.15.131:0/2214808348 wait complete. 2024-04-28T12:37:39.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.668+0000 7fc7ff200640 1 Processor -- start 2024-04-28T12:37:39.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.669+0000 7fc7ff200640 1 -- start start 2024-04-28T12:37:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.669+0000 7fc7ff200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f80cf820 0x7fc7f81f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.669+0000 7fc7ff200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80d01c0 0x7fc7f8140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.669+0000 7fc7ff200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7f806e1b0 0x7fc7f8071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.669+0000 7fc7ff200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f815e600 con 0x7fc7f80cf820 2024-04-28T12:37:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.670+0000 7fc7ff200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f806f8b0 con 0x7fc7f806e1b0 2024-04-28T12:37:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.670+0000 7fc7ff200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f815e780 con 0x7fc7f80d01c0 2024-04-28T12:37:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.670+0000 7fc7f7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f80cf820 0x7fc7f81f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.670+0000 7fc7f7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f80cf820 0x7fc7f81f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47278/0 (socket says 172.21.15.131:47278) 2024-04-28T12:37:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.670+0000 7fc7f7e00640 1 -- 172.21.15.131:0/1888880738 learned_addr learned my addr 172.21.15.131:0/1888880738 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:39.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.670+0000 7fc7ef400640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80d01c0 0x7fc7f8140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:39.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.670+0000 7fc7f7e00640 1 -- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80d01c0 msgr2=0x7fc7f8140d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:39.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.670+0000 7fc7fd200640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7f806e1b0 0x7fc7f8071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:39.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.670+0000 7fc7f7e00640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80d01c0 0x7fc7f8140d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.670+0000 7fc7f7e00640 1 -- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7f806e1b0 msgr2=0x7fc7f8071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:39.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.670+0000 7fc7f7e00640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7f806e1b0 0x7fc7f8071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.670+0000 7fc7f7e00640 1 -- 172.21.15.131:0/1888880738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f81f9370 con 0x7fc7f80cf820 2024-04-28T12:37:39.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.671+0000 7fc7f7e00640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f80cf820 0x7fc7f81f84f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc7e802a670 tx=0x7fc7e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:39.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.671+0000 7fc7f4c00640 1 -- 172.21.15.131:0/1888880738 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8041070 con 0x7fc7f80cf820 2024-04-28T12:37:39.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.671+0000 7fc7f4c00640 1 -- 172.21.15.131:0/1888880738 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e803c670 con 0x7fc7f80cf820 2024-04-28T12:37:39.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.671+0000 7fc7f4c00640 1 -- 172.21.15.131:0/1888880738 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8037040 con 0x7fc7f80cf820 2024-04-28T12:37:39.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.671+0000 7fc7ff200640 1 -- 172.21.15.131:0/1888880738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f81f8ad0 con 0x7fc7f80cf820 2024-04-28T12:37:39.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.671+0000 7fc7ff200640 1 -- 172.21.15.131:0/1888880738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f81f9090 con 0x7fc7f80cf820 2024-04-28T12:37:39.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.672+0000 7fc7ff200640 1 -- 172.21.15.131:0/1888880738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f8069350 con 0x7fc7f80cf820 2024-04-28T12:37:39.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.673+0000 7fc7f4c00640 1 -- 172.21.15.131:0/1888880738 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7e8044070 con 0x7fc7f80cf820 2024-04-28T12:37:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.674+0000 7fc7f4c00640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7d406c8f0 0x7fc7d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.674+0000 7fc7f4c00640 1 -- 172.21.15.131:0/1888880738 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc7e80ac2c0 con 0x7fc7f80cf820 2024-04-28T12:37:39.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.675+0000 7fc7ef400640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7d406c8f0 0x7fc7d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:39.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.676+0000 7fc7ef400640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7d406c8f0 0x7fc7d406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc7f81f2af0 tx=0x7fc7e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:39.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.679+0000 7fc7f4c00640 1 -- 172.21.15.131:0/1888880738 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc7e807a3b0 con 0x7fc7f80cf820 2024-04-28T12:37:39.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.880+0000 7fc7ff200640 1 -- 172.21.15.131:0/1888880738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7f80d1960 con 0x7fc7f80cf820 2024-04-28T12:37:39.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.881+0000 7fc7f4c00640 1 -- 172.21.15.131:0/1888880738 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fc7e8079cb0 con 0x7fc7f80cf820 2024-04-28T12:37:39.877 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:37:39.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.883+0000 7fc7ff200640 1 -- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7d406c8f0 msgr2=0x7fc7d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:39.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.883+0000 7fc7ff200640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7d406c8f0 0x7fc7d406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc7f81f2af0 tx=0x7fc7e0008040 comp rx=0 tx=0).stop 2024-04-28T12:37:39.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.883+0000 7fc7ff200640 1 -- 172.21.15.131:0/1888880738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f80cf820 msgr2=0x7fc7f81f84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:39.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.883+0000 7fc7ff200640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f80cf820 0x7fc7f81f84f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc7e802a670 tx=0x7fc7e80046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.883+0000 7fc7ff200640 1 -- 172.21.15.131:0/1888880738 shutdown_connections 2024-04-28T12:37:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.883+0000 7fc7ff200640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7d406c8f0 0x7fc7d406edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.883+0000 7fc7ff200640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7f806e1b0 0x7fc7f8071ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.883+0000 7fc7ff200640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80d01c0 0x7fc7f8140d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.883+0000 7fc7ff200640 1 --2- 172.21.15.131:0/1888880738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f80cf820 0x7fc7f81f84f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.883+0000 7fc7ff200640 1 -- 172.21.15.131:0/1888880738 >> 172.21.15.131:0/1888880738 conn(0x7fc7f80b2900 msgr2=0x7fc7f80b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.884+0000 7fc7ff200640 1 -- 172.21.15.131:0/1888880738 shutdown_connections 2024-04-28T12:37:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:39.884+0000 7fc7ff200640 1 -- 172.21.15.131:0/1888880738 wait complete. 2024-04-28T12:37:39.898 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:37:40.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.096+0000 7f4608400640 1 Processor -- start 2024-04-28T12:37:40.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.097+0000 7f4608400640 1 -- start start 2024-04-28T12:37:40.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.098+0000 7f4608400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f460014d200 0x7f4600163b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.098+0000 7f4608400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46000723f0 0x7f460006ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.098+0000 7f4608400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4600073660 con 0x7f460006f1b0 2024-04-28T12:37:40.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.098+0000 7f4608400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f460006df40 con 0x7f460014d200 2024-04-28T12:37:40.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.098+0000 7f4608400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f460006e190 con 0x7f46000723f0 2024-04-28T12:37:40.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.098+0000 7f4605a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f460014d200 0x7f4600163b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.098+0000 7f4606400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f460006f1b0 0x7f46000b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33786/0 (socket says 172.21.15.131:33786) 2024-04-28T12:37:40.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.098+0000 7f4606400640 1 -- 172.21.15.131:0/4069226824 learned_addr learned my addr 172.21.15.131:0/4069226824 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:40.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.098+0000 7f4605000640 1 --2- 172.21.15.131:0/4069226824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46000723f0 0x7f460006ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.099+0000 7f4605a00640 1 -- 172.21.15.131:0/4069226824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46000723f0 msgr2=0x7f460006ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:40.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.099+0000 7f4605a00640 1 --2- 172.21.15.131:0/4069226824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46000723f0 0x7f460006ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.099+0000 7f4605a00640 1 -- 172.21.15.131:0/4069226824 >> v1:172.21.15.131:6789/0 conn(0x7f460006f1b0 legacy=0x7f46000b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.099+0000 7f4605a00640 1 -- 172.21.15.131:0/4069226824 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460006e470 con 0x7f460014d200 2024-04-28T12:37:40.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.099+0000 7f45ffe00640 1 -- 172.21.15.131:0/4069226824 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 157392509 0 0) 0x7f4600073660 con 0x7f460006f1b0 2024-04-28T12:37:40.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.100+0000 7f4605a00640 1 --2- 172.21.15.131:0/4069226824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f460014d200 0x7f4600163b70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f45f40097a0 tx=0x7f45f402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9ab8f75b62dd4f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:40.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.100+0000 7f45ffe00640 1 -- 172.21.15.131:0/4069226824 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4037070 con 0x7f460014d200 2024-04-28T12:37:40.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.100+0000 7f45ffe00640 1 -- 172.21.15.131:0/4069226824 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f4032440 con 0x7f460014d200 2024-04-28T12:37:40.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.100+0000 7f45ffe00640 1 -- 172.21.15.131:0/4069226824 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4032810 con 0x7f460014d200 2024-04-28T12:37:40.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.101+0000 7f4608400640 1 -- 172.21.15.131:0/4069226824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f460014d200 msgr2=0x7f4600163b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.101+0000 7f4608400640 1 --2- 172.21.15.131:0/4069226824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f460014d200 0x7f4600163b70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f45f40097a0 tx=0x7f45f402a180 comp rx=0 tx=0).stop 2024-04-28T12:37:40.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.101+0000 7f4608400640 1 -- 172.21.15.131:0/4069226824 shutdown_connections 2024-04-28T12:37:40.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.101+0000 7f4608400640 1 --2- 172.21.15.131:0/4069226824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46000723f0 0x7f460006ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.101+0000 7f4608400640 1 --2- 172.21.15.131:0/4069226824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f460014d200 0x7f4600163b70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.101+0000 7f4608400640 1 -- 172.21.15.131:0/4069226824 >> 172.21.15.131:0/4069226824 conn(0x7f46000b2900 msgr2=0x7f46000b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:40.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.101+0000 7f4608400640 1 -- 172.21.15.131:0/4069226824 shutdown_connections 2024-04-28T12:37:40.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.101+0000 7f4608400640 1 -- 172.21.15.131:0/4069226824 wait complete. 2024-04-28T12:37:40.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.103+0000 7f4608400640 1 Processor -- start 2024-04-28T12:37:40.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.103+0000 7f4608400640 1 -- start start 2024-04-28T12:37:40.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4608400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460006f1b0 0x7f46000c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4608400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46000723f0 0x7f4600148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4608400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460014d200 0x7f4600148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4608400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4600073440 con 0x7f46000723f0 2024-04-28T12:37:40.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4608400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46000732c0 con 0x7f460014d200 2024-04-28T12:37:40.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4608400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f460006f640 con 0x7f460006f1b0 2024-04-28T12:37:40.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4605000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46000723f0 0x7f4600148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4605000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46000723f0 0x7f4600148a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47290/0 (socket says 172.21.15.131:47290) 2024-04-28T12:37:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4605000640 1 -- 172.21.15.131:0/309139967 learned_addr learned my addr 172.21.15.131:0/309139967 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4605a00640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460006f1b0 0x7f46000c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4606400640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460014d200 0x7f4600148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4605000640 1 -- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460006f1b0 msgr2=0x7f46000c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4605000640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460006f1b0 0x7f46000c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.104+0000 7f4605000640 1 -- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460014d200 msgr2=0x7f4600148f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.105+0000 7f4605000640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460014d200 0x7f4600148f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.105+0000 7f4605000640 1 -- 172.21.15.131:0/309139967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46000ad360 con 0x7f46000723f0 2024-04-28T12:37:40.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.105+0000 7f4605000640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46000723f0 0x7f4600148a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45f000d800 tx=0x7f45f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:40.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.105+0000 7f45fd600640 1 -- 172.21.15.131:0/309139967 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f000b6c0 con 0x7f46000723f0 2024-04-28T12:37:40.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.105+0000 7f45fd600640 1 -- 172.21.15.131:0/309139967 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f0019070 con 0x7f46000723f0 2024-04-28T12:37:40.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.105+0000 7f45fd600640 1 -- 172.21.15.131:0/309139967 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f000ba80 con 0x7f46000723f0 2024-04-28T12:37:40.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.105+0000 7f4608400640 1 -- 172.21.15.131:0/309139967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46000ae5e0 con 0x7f46000723f0 2024-04-28T12:37:40.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.105+0000 7f4608400640 1 -- 172.21.15.131:0/309139967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46000aeba0 con 0x7f46000723f0 2024-04-28T12:37:40.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.106+0000 7f4608400640 1 -- 172.21.15.131:0/309139967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4600069350 con 0x7f46000723f0 2024-04-28T12:37:40.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.107+0000 7f45fd600640 1 -- 172.21.15.131:0/309139967 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45f000bc20 con 0x7f46000723f0 2024-04-28T12:37:40.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.108+0000 7f45fd600640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f45e406c8f0 0x7f45e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.109+0000 7f45fd600640 1 -- 172.21.15.131:0/309139967 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f45f008c380 con 0x7f46000723f0 2024-04-28T12:37:40.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.109+0000 7f4605a00640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f45e406c8f0 0x7f45e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.110+0000 7f4605a00640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f45e406c8f0 0x7f45e406edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45f4034000 tx=0x7f45f402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:40.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.112+0000 7f45fd600640 1 -- 172.21.15.131:0/309139967 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45f001c360 con 0x7f46000723f0 2024-04-28T12:37:40.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.311+0000 7f4608400640 1 -- 172.21.15.131:0/309139967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f46000b1280 con 0x7f46000723f0 2024-04-28T12:37:40.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.312+0000 7f45fd600640 1 -- 172.21.15.131:0/309139967 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f45f005a470 con 0x7f46000723f0 2024-04-28T12:37:40.309 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:37:40.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.314+0000 7f4608400640 1 -- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f45e406c8f0 msgr2=0x7f45e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.314+0000 7f4608400640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f45e406c8f0 0x7f45e406edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45f4034000 tx=0x7f45f402e000 comp rx=0 tx=0).stop 2024-04-28T12:37:40.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.314+0000 7f4608400640 1 -- 172.21.15.131:0/309139967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46000723f0 msgr2=0x7f4600148a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.314+0000 7f4608400640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46000723f0 0x7f4600148a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45f000d800 tx=0x7f45f000dcc0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.314+0000 7f4608400640 1 -- 172.21.15.131:0/309139967 shutdown_connections 2024-04-28T12:37:40.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.314+0000 7f4608400640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f45e406c8f0 0x7f45e406edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.314+0000 7f4608400640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f460014d200 0x7f4600148f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.314+0000 7f4608400640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46000723f0 0x7f4600148a00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.314+0000 7f4608400640 1 --2- 172.21.15.131:0/309139967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f460006f1b0 0x7f46000c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.314+0000 7f4608400640 1 -- 172.21.15.131:0/309139967 >> 172.21.15.131:0/309139967 conn(0x7f46000b2900 msgr2=0x7f46000b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:40.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.315+0000 7f4608400640 1 -- 172.21.15.131:0/309139967 shutdown_connections 2024-04-28T12:37:40.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.315+0000 7f4608400640 1 -- 172.21.15.131:0/309139967 wait complete. 2024-04-28T12:37:40.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:37:40.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.541+0000 7fd950c00640 1 Processor -- start 2024-04-28T12:37:40.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.541+0000 7fd950c00640 1 -- start start 2024-04-28T12:37:40.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.542+0000 7fd950c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15cae0 0x7fd94c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.542+0000 7fd950c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd94c15f480 con 0x7fd94c0d12e0 2024-04-28T12:37:40.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.542+0000 7fd950c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd94c15f760 con 0x7fd94c0b5be0 2024-04-28T12:37:40.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.542+0000 7fd950c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd94c160090 con 0x7fd94c15cae0 2024-04-28T12:37:40.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.543+0000 7fd94aa00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd94c0d12e0 0x7fd94c15c3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33794/0 (socket says 172.21.15.131:33794) 2024-04-28T12:37:40.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.543+0000 7fd94aa00640 1 -- 172.21.15.131:0/881692749 learned_addr learned my addr 172.21.15.131:0/881692749 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:40.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.543+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3491112205 0 0) 0x7fd94c15f480 con 0x7fd94c0d12e0 2024-04-28T12:37:40.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.543+0000 7fd94be00640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15cae0 0x7fd94c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.543+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd930003520 con 0x7fd94c0d12e0 2024-04-28T12:37:40.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.543+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1745903368 0 0) 0x7fd930003520 con 0x7fd94c0d12e0 2024-04-28T12:37:40.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.543+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15cae0 msgr2=0x7fd94c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:40.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.543+0000 7fd94a000640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15cae0 0x7fd94c0b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.543+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 >> v1:172.21.15.157:6789/0 conn(0x7fd94c0b5be0 legacy=0x7fd94c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.543+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c15fa40 con 0x7fd94c0d12e0 2024-04-28T12:37:40.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.543+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fd938001670 con 0x7fd94c0d12e0 2024-04-28T12:37:40.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.543+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 >> v1:172.21.15.131:6789/0 conn(0x7fd94c0d12e0 legacy=0x7fd94c15c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94a000640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd930003a50 0x7fd930003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94a000640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd930004800 0x7fd930004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94a000640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15cae0 0x7fd930005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c160090 con 0x7fd930004800 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd93000bbb0 con 0x7fd930003a50 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd93000bdb0 con 0x7fd94c15cae0 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94aa00640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd930004800 0x7fd930004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94b400640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd930003a50 0x7fd930003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94be00640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15cae0 0x7fd930005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94aa00640 1 -- 172.21.15.131:0/881692749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15cae0 msgr2=0x7fd930005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94aa00640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c15cae0 0x7fd930005270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94aa00640 1 -- 172.21.15.131:0/881692749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd930003a50 msgr2=0x7fd930003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94aa00640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd930003a50 0x7fd930003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94aa00640 1 -- 172.21.15.131:0/881692749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd930003520 con 0x7fd930004800 2024-04-28T12:37:40.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.544+0000 7fd94b400640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd930003a50 0x7fd930003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:40.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.545+0000 7fd94aa00640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd930004800 0x7fd930004c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd93800c240 tx=0x7fd930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46527cc509b22ead server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:40.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.546+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938007cf0 con 0x7fd930004800 2024-04-28T12:37:40.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.546+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9380058d0 con 0x7fd930004800 2024-04-28T12:37:40.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.546+0000 7fd94a000640 1 -- 172.21.15.131:0/881692749 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938005ca0 con 0x7fd930004800 2024-04-28T12:37:40.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.546+0000 7fd950c00640 1 -- 172.21.15.131:0/881692749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd930004800 msgr2=0x7fd930004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.546+0000 7fd950c00640 1 --2- 172.21.15.131:0/881692749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd930004800 0x7fd930004c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd93800c240 tx=0x7fd930005830 comp rx=0 tx=0).stop 2024-04-28T12:37:40.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.546+0000 7fd94be00640 1 -- 172.21.15.131:0/881692749 reap_dead start 2024-04-28T12:37:40.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.546+0000 7fd950c00640 1 -- 172.21.15.131:0/881692749 shutdown_connections 2024-04-28T12:37:40.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.546+0000 7fd950c00640 1 -- 172.21.15.131:0/881692749 >> 172.21.15.131:0/881692749 conn(0x7fd94c0684e0 msgr2=0x7fd94c0688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:40.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.546+0000 7fd950c00640 1 -- 172.21.15.131:0/881692749 shutdown_connections 2024-04-28T12:37:40.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.546+0000 7fd950c00640 1 -- 172.21.15.131:0/881692749 wait complete. 2024-04-28T12:37:40.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.548+0000 7fd950c00640 1 Processor -- start 2024-04-28T12:37:40.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.548+0000 7fd950c00640 1 -- start start 2024-04-28T12:37:40.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.549+0000 7fd950c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd930004800 0x7fd94c14c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.549+0000 7fd950c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c14bc10 0x7fd94c0b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.549+0000 7fd950c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd94c0ae8c0 0x7fd94c0aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.549+0000 7fd950c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c0b6140 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.549+0000 7fd950c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c0b5fc0 con 0x7fd930004800 2024-04-28T12:37:40.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.549+0000 7fd950c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd94c0b62c0 con 0x7fd94c14bc10 2024-04-28T12:37:40.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd94c0ae8c0 0x7fd94c0aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd94c0ae8c0 0x7fd94c0aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47316/0 (socket says 172.21.15.131:47316) 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94be00640 1 -- 172.21.15.131:0/229125888 learned_addr learned my addr 172.21.15.131:0/229125888 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94b400640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd930004800 0x7fd94c14c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94aa00640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c14bc10 0x7fd94c0b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94be00640 1 -- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c14bc10 msgr2=0x7fd94c0b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94be00640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c14bc10 0x7fd94c0b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94be00640 1 -- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd930004800 msgr2=0x7fd94c14c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94be00640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd930004800 0x7fd94c14c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94be00640 1 -- 172.21.15.131:0/229125888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c0b2120 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94aa00640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c14bc10 0x7fd94c0b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94b400640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd930004800 0x7fd94c14c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.550+0000 7fd94be00640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd94c0ae8c0 0x7fd94c0aed40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd94c073f40 tx=0x7fd94400e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.551+0000 7fd943400640 1 -- 172.21.15.131:0/229125888 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94400edf0 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.551+0000 7fd943400640 1 -- 172.21.15.131:0/229125888 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94401a070 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.551+0000 7fd943400640 1 -- 172.21.15.131:0/229125888 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944015650 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.551+0000 7fd950c00640 1 -- 172.21.15.131:0/229125888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c14f2e0 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.551+0000 7fd950c00640 1 -- 172.21.15.131:0/229125888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c14f8a0 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.552+0000 7fd950c00640 1 -- 172.21.15.131:0/229125888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94c0506c0 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.553+0000 7fd943400640 1 -- 172.21.15.131:0/229125888 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd944012030 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.554+0000 7fd943400640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd92c06c8f0 0x7fd92c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.554+0000 7fd943400640 1 -- 172.21.15.131:0/229125888 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd94408c620 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.554+0000 7fd94b400640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd92c06c8f0 0x7fd92c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.556+0000 7fd94b400640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd92c06c8f0 0x7fd92c06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0016f0 tx=0x7fd93c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:40.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.558+0000 7fd943400640 1 -- 172.21.15.131:0/229125888 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd944010040 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.763+0000 7fd950c00640 1 -- 172.21.15.131:0/229125888 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd94c072040 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.764+0000 7fd943400640 1 -- 172.21.15.131:0/229125888 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fd94405a670 con 0x7fd94c0ae8c0 2024-04-28T12:37:40.761 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:37:40.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.766+0000 7fd950c00640 1 -- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd92c06c8f0 msgr2=0x7fd92c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.766+0000 7fd950c00640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd92c06c8f0 0x7fd92c06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0016f0 tx=0x7fd93c009040 comp rx=0 tx=0).stop 2024-04-28T12:37:40.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.766+0000 7fd950c00640 1 -- 172.21.15.131:0/229125888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd94c0ae8c0 msgr2=0x7fd94c0aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.766+0000 7fd950c00640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd94c0ae8c0 0x7fd94c0aed40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd94c073f40 tx=0x7fd94400e510 comp rx=0 tx=0).stop 2024-04-28T12:37:40.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.767+0000 7fd950c00640 1 -- 172.21.15.131:0/229125888 shutdown_connections 2024-04-28T12:37:40.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.767+0000 7fd950c00640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd92c06c8f0 0x7fd92c06edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.767+0000 7fd950c00640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd94c0ae8c0 0x7fd94c0aed40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.767+0000 7fd950c00640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd94c14bc10 0x7fd94c0b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.767+0000 7fd950c00640 1 --2- 172.21.15.131:0/229125888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd930004800 0x7fd94c14c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.767+0000 7fd950c00640 1 -- 172.21.15.131:0/229125888 >> 172.21.15.131:0/229125888 conn(0x7fd94c0684e0 msgr2=0x7fd94c0b51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:40.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.767+0000 7fd950c00640 1 -- 172.21.15.131:0/229125888 shutdown_connections 2024-04-28T12:37:40.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.767+0000 7fd950c00640 1 -- 172.21.15.131:0/229125888 wait complete. 2024-04-28T12:37:40.782 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:37:40.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.976+0000 7f2ced600640 1 Processor -- start 2024-04-28T12:37:40.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.976+0000 7f2ced600640 1 -- start start 2024-04-28T12:37:40.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.978+0000 7f2ced600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2ce81458e0 0x7f2ce81604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.978+0000 7f2ced600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ce80b54b0 0x7f2ce815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.978+0000 7f2ced600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ce80b3d80 con 0x7f2ce806df40 2024-04-28T12:37:40.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.978+0000 7f2ced600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2ce80b4060 con 0x7f2ce81458e0 2024-04-28T12:37:40.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.978+0000 7f2ced600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2ce80b4340 con 0x7f2ce80b54b0 2024-04-28T12:37:40.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.978+0000 7f2ce6a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2ce81458e0 0x7f2ce81604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.978+0000 7f2ce6000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2ce806df40 0x7f2ce80b4da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40238/0 (socket says 172.21.15.131:40238) 2024-04-28T12:37:40.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.978+0000 7f2ce6000640 1 -- 172.21.15.131:0/3562482251 learned_addr learned my addr 172.21.15.131:0/3562482251 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:40.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.979+0000 7f2ce7400640 1 --2- 172.21.15.131:0/3562482251 >> v2:172.21.15.157:3300/0 conn(0x7f2ce80b54b0 0x7f2ce815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.979+0000 7f2ce6a00640 1 -- 172.21.15.131:0/3562482251 >> v2:172.21.15.157:3300/0 conn(0x7f2ce80b54b0 msgr2=0x7f2ce815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:40.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.979+0000 7f2ce6a00640 1 --2- 172.21.15.131:0/3562482251 >> v2:172.21.15.157:3300/0 conn(0x7f2ce80b54b0 0x7f2ce815d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.979+0000 7f2ce6a00640 1 -- 172.21.15.131:0/3562482251 >> v1:172.21.15.131:6789/0 conn(0x7f2ce806df40 legacy=0x7f2ce80b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.979+0000 7f2ce6a00640 1 -- 172.21.15.131:0/3562482251 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce80b4620 con 0x7f2ce81458e0 2024-04-28T12:37:40.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.979+0000 7f2ce5600640 1 -- 172.21.15.131:0/3562482251 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1127954197 0 0) 0x7f2ce80b3d80 con 0x7f2ce806df40 2024-04-28T12:37:40.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.980+0000 7f2ce6a00640 1 --2- 172.21.15.131:0/3562482251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce81458e0 0x7f2ce81604d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ce806ddf0 tx=0x7f2cd802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580fde8e137a5ff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.980+0000 7f2ce5600640 1 -- 172.21.15.131:0/3562482251 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8037070 con 0x7f2ce81458e0 2024-04-28T12:37:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.980+0000 7f2ce5600640 1 -- 172.21.15.131:0/3562482251 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd8032440 con 0x7f2ce81458e0 2024-04-28T12:37:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.980+0000 7f2ce5600640 1 -- 172.21.15.131:0/3562482251 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd802ae60 con 0x7f2ce81458e0 2024-04-28T12:37:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.981+0000 7f2ced600640 1 -- 172.21.15.131:0/3562482251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce81458e0 msgr2=0x7f2ce81604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.981+0000 7f2ced600640 1 --2- 172.21.15.131:0/3562482251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce81458e0 0x7f2ce81604d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ce806ddf0 tx=0x7f2cd802a110 comp rx=0 tx=0).stop 2024-04-28T12:37:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.981+0000 7f2ced600640 1 -- 172.21.15.131:0/3562482251 shutdown_connections 2024-04-28T12:37:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.981+0000 7f2ced600640 1 --2- 172.21.15.131:0/3562482251 >> v2:172.21.15.157:3300/0 conn(0x7f2ce80b54b0 0x7f2ce815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.981+0000 7f2ced600640 1 --2- 172.21.15.131:0/3562482251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce81458e0 0x7f2ce81604d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.981+0000 7f2ced600640 1 -- 172.21.15.131:0/3562482251 >> 172.21.15.131:0/3562482251 conn(0x7f2ce80b2900 msgr2=0x7f2ce80b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.981+0000 7f2ced600640 1 -- 172.21.15.131:0/3562482251 shutdown_connections 2024-04-28T12:37:40.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.981+0000 7f2ced600640 1 -- 172.21.15.131:0/3562482251 wait complete. 2024-04-28T12:37:40.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.983+0000 7f2ced600640 1 Processor -- start 2024-04-28T12:37:40.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.983+0000 7f2ced600640 1 -- start start 2024-04-28T12:37:40.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.984+0000 7f2ced600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ce806df40 0x7f2ce8141200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.984+0000 7f2ced600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce80b54b0 0x7f2ce813c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.984+0000 7f2ced600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce81458e0 0x7f2ce813cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.984+0000 7f2ced600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce815ddc0 con 0x7f2ce80b54b0 2024-04-28T12:37:40.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.984+0000 7f2ced600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce815dc40 con 0x7f2ce81458e0 2024-04-28T12:37:40.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.984+0000 7f2ced600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce815df40 con 0x7f2ce806df40 2024-04-28T12:37:40.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.984+0000 7f2ce6000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce80b54b0 0x7f2ce813c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.984+0000 7f2ce6000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce80b54b0 0x7f2ce813c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33372/0 (socket says 172.21.15.131:33372) 2024-04-28T12:37:40.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.984+0000 7f2ce6000640 1 -- 172.21.15.131:0/1700173505 learned_addr learned my addr 172.21.15.131:0/1700173505 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:40.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.984+0000 7f2ce7400640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce81458e0 0x7f2ce813cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.984+0000 7f2ce6000640 1 -- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ce806df40 msgr2=0x7f2ce8141200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.984+0000 7f2ce6000640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ce806df40 0x7f2ce8141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.985+0000 7f2ce6000640 1 -- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce81458e0 msgr2=0x7f2ce813cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:40.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.985+0000 7f2ce6000640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce81458e0 0x7f2ce813cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:40.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.985+0000 7f2ce6000640 1 -- 172.21.15.131:0/1700173505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce80b07c0 con 0x7f2ce80b54b0 2024-04-28T12:37:40.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.985+0000 7f2ce6000640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce80b54b0 0x7f2ce813c8a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000c8c0 tx=0x7f2cd000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.985+0000 7f2cdea00640 1 -- 172.21.15.131:0/1700173505 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0018070 con 0x7f2ce80b54b0 2024-04-28T12:37:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.985+0000 7f2cdea00640 1 -- 172.21.15.131:0/1700173505 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd000a640 con 0x7f2ce80b54b0 2024-04-28T12:37:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.985+0000 7f2cdea00640 1 -- 172.21.15.131:0/1700173505 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd000aa30 con 0x7f2ce80b54b0 2024-04-28T12:37:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.985+0000 7f2ced600640 1 -- 172.21.15.131:0/1700173505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce80b0a60 con 0x7f2ce80b54b0 2024-04-28T12:37:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.985+0000 7f2ced600640 1 -- 172.21.15.131:0/1700173505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce8073f50 con 0x7f2ce80b54b0 2024-04-28T12:37:40.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.986+0000 7f2ced600640 1 -- 172.21.15.131:0/1700173505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce8069350 con 0x7f2ce80b54b0 2024-04-28T12:37:40.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.987+0000 7f2cdea00640 1 -- 172.21.15.131:0/1700173505 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cd001b070 con 0x7f2ce80b54b0 2024-04-28T12:37:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.988+0000 7f2cdea00640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2cc406c8f0 0x7f2cc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.988+0000 7f2cdea00640 1 -- 172.21.15.131:0/1700173505 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2cd008b570 con 0x7f2ce80b54b0 2024-04-28T12:37:40.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.989+0000 7f2ce6a00640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2cc406c8f0 0x7f2cc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:40.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.990+0000 7f2ce6a00640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2cc406c8f0 0x7f2cc406edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8034000 tx=0x7f2cd802c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:40.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:40.993+0000 7f2cdea00640 1 -- 172.21.15.131:0/1700173505 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2cd0059650 con 0x7f2ce80b54b0 2024-04-28T12:37:41.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.193+0000 7f2ced600640 1 -- 172.21.15.131:0/1700173505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2ce806fac0 con 0x7f2ce80b54b0 2024-04-28T12:37:41.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.194+0000 7f2cdea00640 1 -- 172.21.15.131:0/1700173505 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f2cd0058f50 con 0x7f2ce80b54b0 2024-04-28T12:37:41.191 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:37:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.197+0000 7f2ced600640 1 -- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2cc406c8f0 msgr2=0x7f2cc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.197+0000 7f2ced600640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2cc406c8f0 0x7f2cc406edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8034000 tx=0x7f2cd802c000 comp rx=0 tx=0).stop 2024-04-28T12:37:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.197+0000 7f2ced600640 1 -- 172.21.15.131:0/1700173505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce80b54b0 msgr2=0x7f2ce813c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.197+0000 7f2ced600640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce80b54b0 0x7f2ce813c8a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000c8c0 tx=0x7f2cd000cd80 comp rx=0 tx=0).stop 2024-04-28T12:37:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.197+0000 7f2ced600640 1 -- 172.21.15.131:0/1700173505 shutdown_connections 2024-04-28T12:37:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.197+0000 7f2ced600640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2cc406c8f0 0x7f2cc406edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.197+0000 7f2ced600640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ce81458e0 0x7f2ce813cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.197+0000 7f2ced600640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ce80b54b0 0x7f2ce813c8a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.197+0000 7f2ced600640 1 --2- 172.21.15.131:0/1700173505 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ce806df40 0x7f2ce8141200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.197+0000 7f2ced600640 1 -- 172.21.15.131:0/1700173505 >> 172.21.15.131:0/1700173505 conn(0x7f2ce80b2900 msgr2=0x7f2ce806f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:41.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.197+0000 7f2ced600640 1 -- 172.21.15.131:0/1700173505 shutdown_connections 2024-04-28T12:37:41.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.197+0000 7f2ced600640 1 -- 172.21.15.131:0/1700173505 wait complete. 2024-04-28T12:37:41.212 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T12:37:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.401+0000 7fc50b200640 1 Processor -- start 2024-04-28T12:37:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.401+0000 7fc50b200640 1 -- start start 2024-04-28T12:37:41.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc50b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc50c0cd180 0x7fc50c072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:41.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc50b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc50c15dc50 0x7fc50c0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:41.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc50b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc50c0ce2b0 0x7fc50c1607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:41.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc50b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc50c0ce720 con 0x7fc50c15dc50 2024-04-28T12:37:41.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc50b200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc50c0cd560 con 0x7fc50c0cd180 2024-04-28T12:37:41.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc50b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc50c072d00 con 0x7fc50c0ce2b0 2024-04-28T12:37:41.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc503e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc50c15dc50 0x7fc50c0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:41.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc503e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc50c15dc50 0x7fc50c0cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33386/0 (socket says 172.21.15.131:33386) 2024-04-28T12:37:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc503e00640 1 -- 172.21.15.131:0/3809717426 learned_addr learned my addr 172.21.15.131:0/3809717426 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc50a800640 1 --2- 172.21.15.131:0/3809717426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc50c0ce2b0 0x7fc50c1607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc509e00640 1 --2- 172.21.15.131:0/3809717426 >> v2:172.21.15.157:3300/0 conn(0x7fc50c0cd180 0x7fc50c072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc503e00640 1 -- 172.21.15.131:0/3809717426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc50c0ce2b0 msgr2=0x7fc50c1607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc503e00640 1 --2- 172.21.15.131:0/3809717426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc50c0ce2b0 0x7fc50c1607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc503e00640 1 -- 172.21.15.131:0/3809717426 >> v2:172.21.15.157:3300/0 conn(0x7fc50c0cd180 msgr2=0x7fc50c072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc503e00640 1 --2- 172.21.15.131:0/3809717426 >> v2:172.21.15.157:3300/0 conn(0x7fc50c0cd180 0x7fc50c072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc503e00640 1 -- 172.21.15.131:0/3809717426 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c072fe0 con 0x7fc50c15dc50 2024-04-28T12:37:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.403+0000 7fc509e00640 1 --2- 172.21.15.131:0/3809717426 >> v2:172.21.15.157:3300/0 conn(0x7fc50c0cd180 0x7fc50c072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.404+0000 7fc503e00640 1 --2- 172.21.15.131:0/3809717426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c15dc50 0x7fc50c0cdd70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0717f0 tx=0x7fc4f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64980440de1feb92 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.405+0000 7fc509400640 1 -- 172.21.15.131:0/3809717426 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4037070 con 0x7fc50c15dc50 2024-04-28T12:37:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.405+0000 7fc509400640 1 -- 172.21.15.131:0/3809717426 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f4032440 con 0x7fc50c15dc50 2024-04-28T12:37:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.405+0000 7fc509400640 1 -- 172.21.15.131:0/3809717426 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4032810 con 0x7fc50c15dc50 2024-04-28T12:37:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.405+0000 7fc50b200640 1 -- 172.21.15.131:0/3809717426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c15dc50 msgr2=0x7fc50c0cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.405+0000 7fc50b200640 1 --2- 172.21.15.131:0/3809717426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c15dc50 0x7fc50c0cdd70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0717f0 tx=0x7fc4f402a1e0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.405+0000 7fc50b200640 1 -- 172.21.15.131:0/3809717426 shutdown_connections 2024-04-28T12:37:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.405+0000 7fc50b200640 1 --2- 172.21.15.131:0/3809717426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c15dc50 0x7fc50c0cdd70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.405+0000 7fc50b200640 1 --2- 172.21.15.131:0/3809717426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc50c0ce2b0 0x7fc50c1607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.405+0000 7fc50b200640 1 --2- 172.21.15.131:0/3809717426 >> v2:172.21.15.157:3300/0 conn(0x7fc50c0cd180 0x7fc50c072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.405+0000 7fc50b200640 1 -- 172.21.15.131:0/3809717426 >> 172.21.15.131:0/3809717426 conn(0x7fc50c068090 msgr2=0x7fc50c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.406+0000 7fc50b200640 1 -- 172.21.15.131:0/3809717426 shutdown_connections 2024-04-28T12:37:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.406+0000 7fc50b200640 1 -- 172.21.15.131:0/3809717426 wait complete. 2024-04-28T12:37:41.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.407+0000 7fc50b200640 1 Processor -- start 2024-04-28T12:37:41.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.407+0000 7fc50b200640 1 -- start start 2024-04-28T12:37:41.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.408+0000 7fc50b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c0cd180 0x7fc50c159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:41.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.408+0000 7fc50b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc50c0ce2b0 0x7fc50c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:41.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.408+0000 7fc50b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc50c15dc50 0x7fc50c1f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:41.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.408+0000 7fc50b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c073da0 con 0x7fc50c0cd180 2024-04-28T12:37:41.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.408+0000 7fc50b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c0735a0 con 0x7fc50c0ce2b0 2024-04-28T12:37:41.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.408+0000 7fc50b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc50c15e0d0 con 0x7fc50c15dc50 2024-04-28T12:37:41.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.408+0000 7fc509e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c0cd180 0x7fc50c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:41.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.408+0000 7fc509e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c0cd180 0x7fc50c159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33388/0 (socket says 172.21.15.131:33388) 2024-04-28T12:37:41.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.408+0000 7fc509e00640 1 -- 172.21.15.131:0/93088813 learned_addr learned my addr 172.21.15.131:0/93088813 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.409+0000 7fc509e00640 1 -- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc50c15dc50 msgr2=0x7fc50c1f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.409+0000 7fc509e00640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc50c15dc50 0x7fc50c1f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.409+0000 7fc509e00640 1 -- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc50c0ce2b0 msgr2=0x7fc50c15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.409+0000 7fc503e00640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc50c0ce2b0 0x7fc50c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.409+0000 7fc509e00640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc50c0ce2b0 0x7fc50c15f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.409+0000 7fc509e00640 1 -- 172.21.15.131:0/93088813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c13ee70 con 0x7fc50c0cd180 2024-04-28T12:37:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.409+0000 7fc509e00640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c0cd180 0x7fc50c159630 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00c7c0 tx=0x7fc4fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.409+0000 7fc502000640 1 -- 172.21.15.131:0/93088813 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc015070 con 0x7fc50c0cd180 2024-04-28T12:37:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.409+0000 7fc502000640 1 -- 172.21.15.131:0/93088813 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4fc007a30 con 0x7fc50c0cd180 2024-04-28T12:37:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.409+0000 7fc502000640 1 -- 172.21.15.131:0/93088813 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc007e00 con 0x7fc50c0cd180 2024-04-28T12:37:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.409+0000 7fc50b200640 1 -- 172.21.15.131:0/93088813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50c13f3c0 con 0x7fc50c0cd180 2024-04-28T12:37:41.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.410+0000 7fc50b200640 1 -- 172.21.15.131:0/93088813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc50c1600f0 con 0x7fc50c0cd180 2024-04-28T12:37:41.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.411+0000 7fc50b200640 1 -- 172.21.15.131:0/93088813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc50c069350 con 0x7fc50c0cd180 2024-04-28T12:37:41.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.411+0000 7fc502000640 1 -- 172.21.15.131:0/93088813 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4fc018070 con 0x7fc50c0cd180 2024-04-28T12:37:41.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.413+0000 7fc502000640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4e806c8f0 0x7fc4e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:41.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.413+0000 7fc502000640 1 -- 172.21.15.131:0/93088813 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc4fc08be10 con 0x7fc50c0cd180 2024-04-28T12:37:41.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.413+0000 7fc503e00640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4e806c8f0 0x7fc4e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:41.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.414+0000 7fc503e00640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4e806c8f0 0x7fc4e806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4034000 tx=0x7fc4f402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:41.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.417+0000 7fc502000640 1 -- 172.21.15.131:0/93088813 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4fc08f030 con 0x7fc50c0cd180 2024-04-28T12:37:41.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.627+0000 7fc50b200640 1 -- 172.21.15.131:0/93088813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc50c0d0670 con 0x7fc50c0cd180 2024-04-28T12:37:41.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.628+0000 7fc502000640 1 -- 172.21.15.131:0/93088813 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fc4fc059e60 con 0x7fc50c0cd180 2024-04-28T12:37:41.624 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-04-28T12:37:41.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.631+0000 7fc50b200640 1 -- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4e806c8f0 msgr2=0x7fc4e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:41.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.631+0000 7fc50b200640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4e806c8f0 0x7fc4e806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4034000 tx=0x7fc4f402e000 comp rx=0 tx=0).stop 2024-04-28T12:37:41.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.631+0000 7fc50b200640 1 -- 172.21.15.131:0/93088813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c0cd180 msgr2=0x7fc50c159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:41.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.631+0000 7fc50b200640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c0cd180 0x7fc50c159630 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00c7c0 tx=0x7fc4fc00cc80 comp rx=0 tx=0).stop 2024-04-28T12:37:41.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.631+0000 7fc50b200640 1 -- 172.21.15.131:0/93088813 shutdown_connections 2024-04-28T12:37:41.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.631+0000 7fc50b200640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4e806c8f0 0x7fc4e806edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.631+0000 7fc50b200640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc50c15dc50 0x7fc50c1f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.631+0000 7fc50b200640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc50c0ce2b0 0x7fc50c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.631+0000 7fc50b200640 1 --2- 172.21.15.131:0/93088813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c0cd180 0x7fc50c159630 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:41.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.631+0000 7fc50b200640 1 -- 172.21.15.131:0/93088813 >> 172.21.15.131:0/93088813 conn(0x7fc50c068090 msgr2=0x7fc50c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:41.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.631+0000 7fc50b200640 1 -- 172.21.15.131:0/93088813 shutdown_connections 2024-04-28T12:37:41.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:41.631+0000 7fc50b200640 1 -- 172.21.15.131:0/93088813 wait complete. 2024-04-28T12:37:41.646 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-04-28T12:37:41.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:41.646 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:37:41.713 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:ceph-fuse[39892]: fuse finished with error 0 and tester_r 0 2024-04-28T12:37:41.717 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:37:45.401 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:37:45.409 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:37:47.819 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:37:47.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:37:47.820 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-28T12:37:47.896 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:48.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.107+0000 7fc44a000640 1 Processor -- start 2024-04-28T12:37:48.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.107+0000 7fc44a000640 1 -- start start 2024-04-28T12:37:48.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.109+0000 7fc44a000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc4440b50b0 0x7fc4440cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.109+0000 7fc44a000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc44415f2f0 con 0x7fc4440cf7e0 2024-04-28T12:37:48.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.109+0000 7fc44a000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc44406e6c0 con 0x7fc44415ef10 2024-04-28T12:37:48.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.109+0000 7fc44a000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc4440cfc30 con 0x7fc4440b50b0 2024-04-28T12:37:48.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.109+0000 7fc449600640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc4440cf7e0 0x7fc4440b7bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40250/0 (socket says 172.21.15.131:40250) 2024-04-28T12:37:48.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.109+0000 7fc449600640 1 -- 172.21.15.131:0/1072856674 learned_addr learned my addr 172.21.15.131:0/1072856674 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.109+0000 7fc43be00640 1 --2- 172.21.15.131:0/1072856674 >> v2:172.21.15.157:3300/0 conn(0x7fc4440b50b0 0x7fc4440cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.109+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3567139315 0 0) 0x7fc44415f2f0 con 0x7fc4440cf7e0 2024-04-28T12:37:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.109+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc428003520 con 0x7fc4440cf7e0 2024-04-28T12:37:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.109+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 920648386 0 0) 0x7fc428003520 con 0x7fc4440cf7e0 2024-04-28T12:37:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.109+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 >> v2:172.21.15.157:3300/0 conn(0x7fc4440b50b0 msgr2=0x7fc4440cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc443e00640 1 --2- 172.21.15.131:0/1072856674 >> v2:172.21.15.157:3300/0 conn(0x7fc4440b50b0 0x7fc4440cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 >> v1:172.21.15.157:6789/0 conn(0x7fc44415ef10 legacy=0x7fc44406de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4440cd110 con 0x7fc4440cf7e0 2024-04-28T12:37:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fc43c0019f0 con 0x7fc4440cf7e0 2024-04-28T12:37:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 >> v1:172.21.15.131:6789/0 conn(0x7fc4440cf7e0 legacy=0x7fc4440b7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc443e00640 1 --2- 172.21.15.131:0/1072856674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280039e0 0x7fc428003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc443e00640 1 --2- 172.21.15.131:0/1072856674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc428004800 0x7fc428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc443e00640 1 --2- 172.21.15.131:0/1072856674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4280051c0 0x7fc42800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc4440cfc30 con 0x7fc428004800 2024-04-28T12:37:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc42800c090 con 0x7fc4280039e0 2024-04-28T12:37:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc42800c290 con 0x7fc4280051c0 2024-04-28T12:37:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc43be00640 1 --2- 172.21.15.131:0/1072856674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc428004800 0x7fc428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc448c00640 1 --2- 172.21.15.131:0/1072856674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280039e0 0x7fc428003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc449600640 1 --2- 172.21.15.131:0/1072856674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4280051c0 0x7fc42800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc43be00640 1 -- 172.21.15.131:0/1072856674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4280051c0 msgr2=0x7fc42800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc43be00640 1 --2- 172.21.15.131:0/1072856674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4280051c0 0x7fc42800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc43be00640 1 -- 172.21.15.131:0/1072856674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280039e0 msgr2=0x7fc428003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc43be00640 1 --2- 172.21.15.131:0/1072856674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4280039e0 0x7fc428003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc43be00640 1 -- 172.21.15.131:0/1072856674 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc428003520 con 0x7fc428004800 2024-04-28T12:37:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc449600640 1 -- 172.21.15.131:0/1072856674 reap_dead start 2024-04-28T12:37:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.111+0000 7fc43be00640 1 --2- 172.21.15.131:0/1072856674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc428004800 0x7fc428004c80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009ac0 tx=0x7fc428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ece3fb5523aa8fe server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.112+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c036070 con 0x7fc428004800 2024-04-28T12:37:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.112+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c031400 con 0x7fc428004800 2024-04-28T12:37:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.112+0000 7fc443e00640 1 -- 172.21.15.131:0/1072856674 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c009690 con 0x7fc428004800 2024-04-28T12:37:48.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.112+0000 7fc44a000640 1 -- 172.21.15.131:0/1072856674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc428004800 msgr2=0x7fc428004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:48.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.112+0000 7fc44a000640 1 --2- 172.21.15.131:0/1072856674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc428004800 0x7fc428004c80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009ac0 tx=0x7fc428005830 comp rx=0 tx=0).stop 2024-04-28T12:37:48.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.112+0000 7fc44a000640 1 -- 172.21.15.131:0/1072856674 shutdown_connections 2024-04-28T12:37:48.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.112+0000 7fc44a000640 1 --2- 172.21.15.131:0/1072856674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc428004800 0x7fc428004c80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.112+0000 7fc44a000640 1 -- 172.21.15.131:0/1072856674 >> 172.21.15.131:0/1072856674 conn(0x7fc4440b4590 msgr2=0x7fc4440b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:48.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.114+0000 7fc44a000640 1 -- 172.21.15.131:0/1072856674 shutdown_connections 2024-04-28T12:37:48.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.114+0000 7fc44a000640 1 -- 172.21.15.131:0/1072856674 wait complete. 2024-04-28T12:37:48.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.115+0000 7fc44a000640 1 Processor -- start 2024-04-28T12:37:48.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.115+0000 7fc44a000640 1 -- start start 2024-04-28T12:37:48.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.116+0000 7fc44a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc428004800 0x7fc4440c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.116+0000 7fc44a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc444149c20 0x7fc4440c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.116+0000 7fc44a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc44414a0a0 0x7fc4440c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.116+0000 7fc44a000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc4440cd6d0 con 0x7fc444149c20 2024-04-28T12:37:48.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.116+0000 7fc44a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4440cda40 con 0x7fc44414a0a0 2024-04-28T12:37:48.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.116+0000 7fc44a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4440b5890 con 0x7fc428004800 2024-04-28T12:37:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.117+0000 7fc43be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc444149c20 0x7fc4440c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.117+0000 7fc43be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc444149c20 0x7fc4440c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33404/0 (socket says 172.21.15.131:33404) 2024-04-28T12:37:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.117+0000 7fc43be00640 1 -- 172.21.15.131:0/629706189 learned_addr learned my addr 172.21.15.131:0/629706189 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.117+0000 7fc448c00640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc428004800 0x7fc4440c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.117+0000 7fc449600640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc44414a0a0 0x7fc4440c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.117+0000 7fc43be00640 1 -- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc428004800 msgr2=0x7fc4440c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.117+0000 7fc43be00640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc428004800 0x7fc4440c65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.117+0000 7fc43be00640 1 -- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc44414a0a0 msgr2=0x7fc4440c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.117+0000 7fc43be00640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc44414a0a0 0x7fc4440c9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.117+0000 7fc43be00640 1 -- 172.21.15.131:0/629706189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4440c6e20 con 0x7fc444149c20 2024-04-28T12:37:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.117+0000 7fc43be00640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc444149c20 0x7fc4440c8b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc42c02abb0 tx=0x7fc42c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:48.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.118+0000 7fc441600640 1 -- 172.21.15.131:0/629706189 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c03f020 con 0x7fc444149c20 2024-04-28T12:37:48.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.118+0000 7fc441600640 1 -- 172.21.15.131:0/629706189 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c037040 con 0x7fc444149c20 2024-04-28T12:37:48.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.118+0000 7fc44a000640 1 -- 172.21.15.131:0/629706189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4441606b0 con 0x7fc444149c20 2024-04-28T12:37:48.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.118+0000 7fc441600640 1 -- 172.21.15.131:0/629706189 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c036070 con 0x7fc444149c20 2024-04-28T12:37:48.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.118+0000 7fc44a000640 1 -- 172.21.15.131:0/629706189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc444160c70 con 0x7fc444149c20 2024-04-28T12:37:48.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.119+0000 7fc44a000640 1 -- 172.21.15.131:0/629706189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc444069350 con 0x7fc444149c20 2024-04-28T12:37:48.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.120+0000 7fc441600640 1 -- 172.21.15.131:0/629706189 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc42c042020 con 0x7fc444149c20 2024-04-28T12:37:48.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.120+0000 7fc441600640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc42006c920 0x7fc42006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.122+0000 7fc441600640 1 -- 172.21.15.131:0/629706189 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc42c0ac3c0 con 0x7fc444149c20 2024-04-28T12:37:48.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.122+0000 7fc448c00640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc42006c920 0x7fc42006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.123+0000 7fc448c00640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc42006c920 0x7fc42006ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc434007fd0 tx=0x7fc434009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:48.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.124+0000 7fc441600640 1 -- 172.21.15.131:0/629706189 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc42c049070 con 0x7fc444149c20 2024-04-28T12:37:48.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.327+0000 7fc44a000640 1 -- 172.21.15.131:0/629706189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4440d24b0 con 0x7fc444149c20 2024-04-28T12:37:48.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.330+0000 7fc441600640 1 -- 172.21.15.131:0/629706189 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7fc42c07a520 con 0x7fc444149c20 2024-04-28T12:37:48.327 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:48.327 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.333+0000 7fc44a000640 1 -- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc42006c920 msgr2=0x7fc42006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.333+0000 7fc44a000640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc42006c920 0x7fc42006ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc434007fd0 tx=0x7fc434009040 comp rx=0 tx=0).stop 2024-04-28T12:37:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.333+0000 7fc44a000640 1 -- 172.21.15.131:0/629706189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc444149c20 msgr2=0x7fc4440c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.333+0000 7fc44a000640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc444149c20 0x7fc4440c8b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc42c02abb0 tx=0x7fc42c004730 comp rx=0 tx=0).stop 2024-04-28T12:37:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.333+0000 7fc44a000640 1 -- 172.21.15.131:0/629706189 shutdown_connections 2024-04-28T12:37:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.333+0000 7fc44a000640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc42006c920 0x7fc42006ede0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.333+0000 7fc44a000640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc44414a0a0 0x7fc4440c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.333+0000 7fc44a000640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc444149c20 0x7fc4440c8b40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.333+0000 7fc44a000640 1 --2- 172.21.15.131:0/629706189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc428004800 0x7fc4440c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.333+0000 7fc44a000640 1 -- 172.21.15.131:0/629706189 >> 172.21.15.131:0/629706189 conn(0x7fc4440b4590 msgr2=0x7fc4440b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:48.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.333+0000 7fc44a000640 1 -- 172.21.15.131:0/629706189 shutdown_connections 2024-04-28T12:37:48.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.333+0000 7fc44a000640 1 -- 172.21.15.131:0/629706189 wait complete. 2024-04-28T12:37:48.349 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:48.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.552+0000 7fc185000640 1 Processor -- start 2024-04-28T12:37:48.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.552+0000 7fc185000640 1 -- start start 2024-04-28T12:37:48.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.554+0000 7fc185000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1800d1620 0x7fc1800d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.554+0000 7fc185000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc1800d1a00 0x7fc1800b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.554+0000 7fc185000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1800d0480 con 0x7fc1800b50b0 2024-04-28T12:37:48.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.554+0000 7fc185000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc1800d0ce0 con 0x7fc1800d1a00 2024-04-28T12:37:48.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.554+0000 7fc185000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc1800d0720 con 0x7fc1800d1620 2024-04-28T12:37:48.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.554+0000 7fc17ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc1800d1a00 0x7fc1800b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.554+0000 7fc17ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc1800d1a00 0x7fc1800b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33410/0 (socket says 172.21.15.131:33410) 2024-04-28T12:37:48.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.554+0000 7fc17ea00640 1 -- 172.21.15.131:0/3945941326 learned_addr learned my addr 172.21.15.131:0/3945941326 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:48.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.554+0000 7fc17f400640 1 --2- 172.21.15.131:0/3945941326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1800d1620 0x7fc1800d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.554+0000 7fc17ea00640 1 -- 172.21.15.131:0/3945941326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1800d1620 msgr2=0x7fc1800d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.554+0000 7fc17ea00640 1 --2- 172.21.15.131:0/3945941326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1800d1620 0x7fc1800d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.555+0000 7fc17ea00640 1 -- 172.21.15.131:0/3945941326 >> v1:172.21.15.157:6789/0 conn(0x7fc1800b50b0 legacy=0x7fc1800b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.555+0000 7fc17ea00640 1 -- 172.21.15.131:0/3945941326 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1800d0a00 con 0x7fc1800d1a00 2024-04-28T12:37:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.555+0000 7fc17ea00640 1 --2- 172.21.15.131:0/3945941326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1800d1a00 0x7fc1800b4b70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009e50 tx=0x7fc16c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be8ae9fa29fbf20 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.556+0000 7fc17e000640 1 -- 172.21.15.131:0/3945941326 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c036070 con 0x7fc1800d1a00 2024-04-28T12:37:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.556+0000 7fc17e000640 1 -- 172.21.15.131:0/3945941326 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc16c031600 con 0x7fc1800d1a00 2024-04-28T12:37:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.556+0000 7fc17e000640 1 -- 172.21.15.131:0/3945941326 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0319d0 con 0x7fc1800d1a00 2024-04-28T12:37:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.556+0000 7fc185000640 1 -- 172.21.15.131:0/3945941326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1800d1a00 msgr2=0x7fc1800b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.556+0000 7fc185000640 1 --2- 172.21.15.131:0/3945941326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1800d1a00 0x7fc1800b4b70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009e50 tx=0x7fc16c02a630 comp rx=0 tx=0).stop 2024-04-28T12:37:48.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.557+0000 7fc185000640 1 -- 172.21.15.131:0/3945941326 shutdown_connections 2024-04-28T12:37:48.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.557+0000 7fc185000640 1 --2- 172.21.15.131:0/3945941326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1800d1a00 0x7fc1800b4b70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.557+0000 7fc185000640 1 --2- 172.21.15.131:0/3945941326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1800d1620 0x7fc1800d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.557+0000 7fc185000640 1 -- 172.21.15.131:0/3945941326 >> 172.21.15.131:0/3945941326 conn(0x7fc1800d4c30 msgr2=0x7fc1800d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:48.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.557+0000 7fc185000640 1 -- 172.21.15.131:0/3945941326 shutdown_connections 2024-04-28T12:37:48.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.557+0000 7fc185000640 1 -- 172.21.15.131:0/3945941326 wait complete. 2024-04-28T12:37:48.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.558+0000 7fc185000640 1 Processor -- start 2024-04-28T12:37:48.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.559+0000 7fc185000640 1 -- start start 2024-04-28T12:37:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.559+0000 7fc185000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1800b50b0 0x7fc1800b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.559+0000 7fc185000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1800d1620 0x7fc1800b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.559+0000 7fc185000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1800d1a00 0x7fc1800b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.559+0000 7fc185000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc1800b5c50 con 0x7fc1800d1a00 2024-04-28T12:37:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc185000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1800d1e60 con 0x7fc1800d1620 2024-04-28T12:37:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc185000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc1800b5dd0 con 0x7fc1800b50b0 2024-04-28T12:37:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc17fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1800d1a00 0x7fc1800b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc17fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1800d1a00 0x7fc1800b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33424/0 (socket says 172.21.15.131:33424) 2024-04-28T12:37:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc17fe00640 1 -- 172.21.15.131:0/3614790728 learned_addr learned my addr 172.21.15.131:0/3614790728 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc17f400640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1800b50b0 0x7fc1800b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc17ea00640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1800d1620 0x7fc1800b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc17fe00640 1 -- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1800b50b0 msgr2=0x7fc1800b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc17fe00640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1800b50b0 0x7fc1800b8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc17fe00640 1 -- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1800d1620 msgr2=0x7fc1800b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc17fe00640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1800d1620 0x7fc1800b7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc17fe00640 1 -- 172.21.15.131:0/3614790728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1800c9880 con 0x7fc1800d1a00 2024-04-28T12:37:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.560+0000 7fc17ea00640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1800d1620 0x7fc1800b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.561+0000 7fc17fe00640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1800d1a00 0x7fc1800b2360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c410 tx=0x7fc17800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:48.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.561+0000 7fc177400640 1 -- 172.21.15.131:0/3614790728 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178011790 con 0x7fc1800d1a00 2024-04-28T12:37:48.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.561+0000 7fc177400640 1 -- 172.21.15.131:0/3614790728 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc178017070 con 0x7fc1800d1a00 2024-04-28T12:37:48.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.561+0000 7fc177400640 1 -- 172.21.15.131:0/3614790728 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178011bb0 con 0x7fc1800d1a00 2024-04-28T12:37:48.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.561+0000 7fc185000640 1 -- 172.21.15.131:0/3614790728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1800c9b60 con 0x7fc1800d1a00 2024-04-28T12:37:48.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.561+0000 7fc185000640 1 -- 172.21.15.131:0/3614790728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1800b77b0 con 0x7fc1800d1a00 2024-04-28T12:37:48.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.562+0000 7fc185000640 1 -- 172.21.15.131:0/3614790728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc180069350 con 0x7fc1800d1a00 2024-04-28T12:37:48.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.563+0000 7fc177400640 1 -- 172.21.15.131:0/3614790728 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc17801c460 con 0x7fc1800d1a00 2024-04-28T12:37:48.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.564+0000 7fc177400640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc16006c920 0x7fc16006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.564+0000 7fc177400640 1 -- 172.21.15.131:0/3614790728 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc17808bac0 con 0x7fc1800d1a00 2024-04-28T12:37:48.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.565+0000 7fc17f400640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc16006c920 0x7fc16006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:48.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.566+0000 7fc17f400640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc16006c920 0x7fc16006ede0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc170006fd0 tx=0x7fc170008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.569+0000 7fc177400640 1 -- 172.21.15.131:0/3614790728 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc178021020 con 0x7fc1800d1a00 2024-04-28T12:37:48.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.771+0000 7fc185000640 1 -- 172.21.15.131:0/3614790728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1800d4770 con 0x7fc1800d1a00 2024-04-28T12:37:48.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.774+0000 7fc177400640 1 -- 172.21.15.131:0/3614790728 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7fc178059b10 con 0x7fc1800d1a00 2024-04-28T12:37:48.772 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:48.772 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:48.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.778+0000 7fc185000640 1 -- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc16006c920 msgr2=0x7fc16006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:48.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.778+0000 7fc185000640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc16006c920 0x7fc16006ede0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc170006fd0 tx=0x7fc170008040 comp rx=0 tx=0).stop 2024-04-28T12:37:48.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.778+0000 7fc185000640 1 -- 172.21.15.131:0/3614790728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1800d1a00 msgr2=0x7fc1800b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:48.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.778+0000 7fc185000640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1800d1a00 0x7fc1800b2360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c410 tx=0x7fc17800c8d0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.778+0000 7fc185000640 1 -- 172.21.15.131:0/3614790728 shutdown_connections 2024-04-28T12:37:48.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.778+0000 7fc185000640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc16006c920 0x7fc16006ede0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.778+0000 7fc185000640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1800d1a00 0x7fc1800b2360 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.778+0000 7fc185000640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1800d1620 0x7fc1800b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.778+0000 7fc185000640 1 --2- 172.21.15.131:0/3614790728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1800b50b0 0x7fc1800b8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:48.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.778+0000 7fc185000640 1 -- 172.21.15.131:0/3614790728 >> 172.21.15.131:0/3614790728 conn(0x7fc1800d4c30 msgr2=0x7fc1800d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:48.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.778+0000 7fc185000640 1 -- 172.21.15.131:0/3614790728 shutdown_connections 2024-04-28T12:37:48.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.778+0000 7fc185000640 1 -- 172.21.15.131:0/3614790728 wait complete. 2024-04-28T12:37:48.794 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-28T12:37:48.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.999+0000 7fd90ba00640 1 Processor -- start 2024-04-28T12:37:48.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:48.999+0000 7fd90ba00640 1 -- start start 2024-04-28T12:37:48.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.001+0000 7fd90ba00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd9040b5250 0x7fd9040cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:48.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.001+0000 7fd90ba00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd90406eaf0 con 0x7fd90415d5f0 2024-04-28T12:37:48.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.001+0000 7fd90ba00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd90406edd0 con 0x7fd9040cf440 2024-04-28T12:37:48.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.001+0000 7fd90ba00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd90406f420 con 0x7fd9040b5250 2024-04-28T12:37:48.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.001+0000 7fd909a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd90415d5f0 0x7fd90415da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40260/0 (socket says 172.21.15.131:40260) 2024-04-28T12:37:48.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.001+0000 7fd909a00640 1 -- 172.21.15.131:0/487602097 learned_addr learned my addr 172.21.15.131:0/487602097 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:48.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.001+0000 7fd903e00640 1 --2- 172.21.15.131:0/487602097 >> v2:172.21.15.157:3300/0 conn(0x7fd9040b5250 0x7fd9040cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1178333534 0 0) 0x7fd90406eaf0 con 0x7fd90415d5f0 2024-04-28T12:37:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8ec003520 con 0x7fd90415d5f0 2024-04-28T12:37:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2384989012 0 0) 0x7fd8ec003520 con 0x7fd90415d5f0 2024-04-28T12:37:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 >> v2:172.21.15.157:3300/0 conn(0x7fd9040b5250 msgr2=0x7fd9040cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 --2- 172.21.15.131:0/487602097 >> v2:172.21.15.157:3300/0 conn(0x7fd9040b5250 0x7fd9040cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 >> v1:172.21.15.157:6789/0 conn(0x7fd9040cf440 legacy=0x7fd9040cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90406f0b0 con 0x7fd90415d5f0 2024-04-28T12:37:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fd8fc0019f0 con 0x7fd90415d5f0 2024-04-28T12:37:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 >> v1:172.21.15.131:6789/0 conn(0x7fd90415d5f0 legacy=0x7fd90415da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 --2- 172.21.15.131:0/487602097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ec0039b0 0x7fd8ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 --2- 172.21.15.131:0/487602097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8ec004800 0x7fd8ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 --2- 172.21.15.131:0/487602097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ec0051c0 0x7fd8ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd90406f420 con 0x7fd8ec0039b0 2024-04-28T12:37:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec00c160 con 0x7fd8ec004800 2024-04-28T12:37:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.002+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec00c360 con 0x7fd8ec0051c0 2024-04-28T12:37:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.003+0000 7fd909000640 1 --2- 172.21.15.131:0/487602097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ec0039b0 0x7fd8ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.003+0000 7fd903e00640 1 --2- 172.21.15.131:0/487602097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8ec004800 0x7fd8ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.003+0000 7fd909000640 1 -- 172.21.15.131:0/487602097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ec0051c0 msgr2=0x7fd8ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.003+0000 7fd909000640 1 --2- 172.21.15.131:0/487602097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ec0051c0 0x7fd8ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.003+0000 7fd909000640 1 -- 172.21.15.131:0/487602097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8ec004800 msgr2=0x7fd8ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.003+0000 7fd909000640 1 --2- 172.21.15.131:0/487602097 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8ec004800 0x7fd8ec004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.003+0000 7fd909000640 1 -- 172.21.15.131:0/487602097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec003520 con 0x7fd8ec0039b0 2024-04-28T12:37:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.003+0000 7fd909a00640 1 -- 172.21.15.131:0/487602097 reap_dead start 2024-04-28T12:37:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.004+0000 7fd909000640 1 --2- 172.21.15.131:0/487602097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ec0039b0 0x7fd8ec003dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8001750 tx=0x7fd8ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c39e9866b6f834 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.004+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8001370 con 0x7fd8ec0039b0 2024-04-28T12:37:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.004+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f80052d0 con 0x7fd8ec0039b0 2024-04-28T12:37:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.004+0000 7fd903400640 1 -- 172.21.15.131:0/487602097 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f802abc0 con 0x7fd8ec0039b0 2024-04-28T12:37:49.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.005+0000 7fd90ba00640 1 -- 172.21.15.131:0/487602097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ec0039b0 msgr2=0x7fd8ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:49.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.005+0000 7fd90ba00640 1 --2- 172.21.15.131:0/487602097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ec0039b0 0x7fd8ec003dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8001750 tx=0x7fd8ec005830 comp rx=0 tx=0).stop 2024-04-28T12:37:49.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.005+0000 7fd90ba00640 1 -- 172.21.15.131:0/487602097 shutdown_connections 2024-04-28T12:37:49.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.005+0000 7fd90ba00640 1 --2- 172.21.15.131:0/487602097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8ec0039b0 0x7fd8ec003dd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:49.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.005+0000 7fd90ba00640 1 -- 172.21.15.131:0/487602097 >> 172.21.15.131:0/487602097 conn(0x7fd9040b2900 msgr2=0x7fd9040b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:49.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.005+0000 7fd90ba00640 1 -- 172.21.15.131:0/487602097 shutdown_connections 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.005+0000 7fd90ba00640 1 -- 172.21.15.131:0/487602097 wait complete. 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.007+0000 7fd90ba00640 1 Processor -- start 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.007+0000 7fd90ba00640 1 -- start start 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.008+0000 7fd90ba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ec0039b0 0x7fd904130fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.008+0000 7fd90ba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd90415ca10 0x7fd9040aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.008+0000 7fd90ba00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd904132640 0x7fd9041344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.008+0000 7fd90ba00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd90415da80 con 0x7fd904132640 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.008+0000 7fd90ba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd90406f8b0 con 0x7fd90415ca10 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.008+0000 7fd90ba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd90415dc00 con 0x7fd8ec0039b0 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.008+0000 7fd909a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd904132640 0x7fd9041344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.008+0000 7fd909a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd904132640 0x7fd9041344b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33434/0 (socket says 172.21.15.131:33434) 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.008+0000 7fd909a00640 1 -- 172.21.15.131:0/3926957822 learned_addr learned my addr 172.21.15.131:0/3926957822 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:49.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.008+0000 7fd909000640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ec0039b0 0x7fd904130fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.008+0000 7fd903e00640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd90415ca10 0x7fd9040aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.009+0000 7fd909a00640 1 -- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ec0039b0 msgr2=0x7fd904130fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.009+0000 7fd909a00640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ec0039b0 0x7fd904130fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.009+0000 7fd909a00640 1 -- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd90415ca10 msgr2=0x7fd9040aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.009+0000 7fd909a00640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd90415ca10 0x7fd9040aaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.009+0000 7fd909a00640 1 -- 172.21.15.131:0/3926957822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9041315a0 con 0x7fd904132640 2024-04-28T12:37:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.009+0000 7fd909a00640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd904132640 0x7fd9041344b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd90406d7e0 tx=0x7fd8fc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.009+0000 7fd900c00640 1 -- 172.21.15.131:0/3926957822 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc004d60 con 0x7fd904132640 2024-04-28T12:37:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.009+0000 7fd900c00640 1 -- 172.21.15.131:0/3926957822 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8fc00a5b0 con 0x7fd904132640 2024-04-28T12:37:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.009+0000 7fd900c00640 1 -- 172.21.15.131:0/3926957822 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc00a9a0 con 0x7fd904132640 2024-04-28T12:37:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.009+0000 7fd90ba00640 1 -- 172.21.15.131:0/3926957822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd904131880 con 0x7fd904132640 2024-04-28T12:37:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.009+0000 7fd90ba00640 1 -- 172.21.15.131:0/3926957822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd904131e40 con 0x7fd904132640 2024-04-28T12:37:49.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.011+0000 7fd90ba00640 1 -- 172.21.15.131:0/3926957822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90406eaf0 con 0x7fd904132640 2024-04-28T12:37:49.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.011+0000 7fd900c00640 1 -- 172.21.15.131:0/3926957822 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8fc011070 con 0x7fd904132640 2024-04-28T12:37:49.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.013+0000 7fd900c00640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd8e806c8f0 0x7fd8e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:49.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.013+0000 7fd900c00640 1 -- 172.21.15.131:0/3926957822 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd8fc08b590 con 0x7fd904132640 2024-04-28T12:37:49.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.013+0000 7fd909000640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd8e806c8f0 0x7fd8e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:49.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.014+0000 7fd909000640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd8e806c8f0 0x7fd8e806edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8032000 tx=0x7fd8f802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.018+0000 7fd900c00640 1 -- 172.21.15.131:0/3926957822 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd90406eaf0 con 0x7fd904132640 2024-04-28T12:37:49.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:49.262+0000 7fd90ba00640 1 -- 172.21.15.131:0/3926957822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd9040b2d80 con 0x7fd904132640 2024-04-28T12:37:50.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.045+0000 7fd900c00640 1 -- 172.21.15.131:0/3926957822 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fd8fc01a060 con 0x7fd904132640 2024-04-28T12:37:50.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.049+0000 7fd90ba00640 1 -- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd8e806c8f0 msgr2=0x7fd8e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:50.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.049+0000 7fd90ba00640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd8e806c8f0 0x7fd8e806edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8032000 tx=0x7fd8f802e000 comp rx=0 tx=0).stop 2024-04-28T12:37:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.049+0000 7fd90ba00640 1 -- 172.21.15.131:0/3926957822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd904132640 msgr2=0x7fd9041344b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.049+0000 7fd90ba00640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd904132640 0x7fd9041344b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd90406d7e0 tx=0x7fd8fc00cc00 comp rx=0 tx=0).stop 2024-04-28T12:37:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.049+0000 7fd90ba00640 1 -- 172.21.15.131:0/3926957822 shutdown_connections 2024-04-28T12:37:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.049+0000 7fd90ba00640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd8e806c8f0 0x7fd8e806edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.049+0000 7fd90ba00640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd904132640 0x7fd9041344b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:50.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.049+0000 7fd90ba00640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd90415ca10 0x7fd9040aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.049+0000 7fd90ba00640 1 --2- 172.21.15.131:0/3926957822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8ec0039b0 0x7fd904130fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.049+0000 7fd90ba00640 1 -- 172.21.15.131:0/3926957822 >> 172.21.15.131:0/3926957822 conn(0x7fd9040b2900 msgr2=0x7fd9040b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.049+0000 7fd90ba00640 1 -- 172.21.15.131:0/3926957822 shutdown_connections 2024-04-28T12:37:50.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.050+0000 7fd90ba00640 1 -- 172.21.15.131:0/3926957822 wait complete. 2024-04-28T12:37:50.070 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-28T12:37:50.070 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-28T12:37:50.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.275+0000 7f1faa400640 1 Processor -- start 2024-04-28T12:37:50.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.276+0000 7f1faa400640 1 -- start start 2024-04-28T12:37:50.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.277+0000 7f1faa400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1fa415cca0 0x7f1fa40cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:50.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.277+0000 7f1faa400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1fa4160240 0x7f1fa40d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:50.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.277+0000 7f1faa400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa40d12e0 con 0x7f1fa40d1d70 2024-04-28T12:37:50.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.277+0000 7f1faa400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa40d21a0 con 0x7f1fa415cca0 2024-04-28T12:37:50.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.277+0000 7f1faa400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa40d0240 con 0x7f1fa4160240 2024-04-28T12:37:50.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.277+0000 7f1fa3400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1fa415cca0 0x7f1fa40cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:50.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.277+0000 7f1fa3400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1fa415cca0 0x7f1fa40cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33440/0 (socket says 172.21.15.131:33440) 2024-04-28T12:37:50.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.277+0000 7f1fa3400640 1 -- 172.21.15.131:0/1913055858 learned_addr learned my addr 172.21.15.131:0/1913055858 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:50.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.277+0000 7f1fa2a00640 1 --2- 172.21.15.131:0/1913055858 >> v2:172.21.15.157:3300/0 conn(0x7f1fa4160240 0x7f1fa40d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:50.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.278+0000 7f1fa3400640 1 -- 172.21.15.131:0/1913055858 >> v2:172.21.15.157:3300/0 conn(0x7f1fa4160240 msgr2=0x7f1fa40d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:50.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.278+0000 7f1fa3400640 1 --2- 172.21.15.131:0/1913055858 >> v2:172.21.15.157:3300/0 conn(0x7f1fa4160240 0x7f1fa40d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:50.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.278+0000 7f1fa3400640 1 -- 172.21.15.131:0/1913055858 >> v1:172.21.15.131:6789/0 conn(0x7f1fa40d1d70 legacy=0x7f1fa4162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:50.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.278+0000 7f1fa3400640 1 -- 172.21.15.131:0/1913055858 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa40d0520 con 0x7f1fa415cca0 2024-04-28T12:37:50.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.279+0000 7f1fa3400640 1 --2- 172.21.15.131:0/1913055858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fa415cca0 0x7f1fa40cfc30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40d0170 tx=0x7f1f9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e682006827cd14a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:50.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.279+0000 7f1fa2000640 1 -- 172.21.15.131:0/1913055858 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94037070 con 0x7f1fa415cca0 2024-04-28T12:37:50.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.279+0000 7f1fa2000640 1 -- 172.21.15.131:0/1913055858 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f94032440 con 0x7f1fa415cca0 2024-04-28T12:37:50.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.279+0000 7f1fa2000640 1 -- 172.21.15.131:0/1913055858 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94032810 con 0x7f1fa415cca0 2024-04-28T12:37:50.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.280+0000 7f1faa400640 1 -- 172.21.15.131:0/1913055858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fa415cca0 msgr2=0x7f1fa40cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:50.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.280+0000 7f1faa400640 1 --2- 172.21.15.131:0/1913055858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fa415cca0 0x7f1fa40cfc30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40d0170 tx=0x7f1f9402a270 comp rx=0 tx=0).stop 2024-04-28T12:37:50.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.280+0000 7f1faa400640 1 -- 172.21.15.131:0/1913055858 shutdown_connections 2024-04-28T12:37:50.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.280+0000 7f1faa400640 1 --2- 172.21.15.131:0/1913055858 >> v2:172.21.15.157:3300/0 conn(0x7f1fa4160240 0x7f1fa40d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:50.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.280+0000 7f1faa400640 1 --2- 172.21.15.131:0/1913055858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fa415cca0 0x7f1fa40cfc30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:50.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.280+0000 7f1faa400640 1 -- 172.21.15.131:0/1913055858 >> 172.21.15.131:0/1913055858 conn(0x7f1fa40d4e40 msgr2=0x7f1fa40d5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:50.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.280+0000 7f1faa400640 1 -- 172.21.15.131:0/1913055858 shutdown_connections 2024-04-28T12:37:50.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.280+0000 7f1faa400640 1 -- 172.21.15.131:0/1913055858 wait complete. 2024-04-28T12:37:50.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.282+0000 7f1faa400640 1 Processor -- start 2024-04-28T12:37:50.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.282+0000 7f1faa400640 1 -- start start 2024-04-28T12:37:50.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1faa400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fa40d1d70 0x7f1fa40ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:50.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1faa400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa415cca0 0x7f1fa41f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:50.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1faa400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa4160240 0x7f1fa41f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:50.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1faa400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa40d0dc0 con 0x7f1fa40d1d70 2024-04-28T12:37:50.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1faa400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa40d0ae0 con 0x7f1fa4160240 2024-04-28T12:37:50.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1faa400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa41606b0 con 0x7f1fa415cca0 2024-04-28T12:37:50.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1fa3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fa40d1d70 0x7f1fa40ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1fa3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fa40d1d70 0x7f1fa40ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33448/0 (socket says 172.21.15.131:33448) 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1fa3400640 1 -- 172.21.15.131:0/2492665519 learned_addr learned my addr 172.21.15.131:0/2492665519 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1fa3400640 1 -- 172.21.15.131:0/2492665519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa415cca0 msgr2=0x7f1fa41f8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1fa3400640 1 --2- 172.21.15.131:0/2492665519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa415cca0 0x7f1fa41f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1fa3400640 1 -- 172.21.15.131:0/2492665519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa4160240 msgr2=0x7f1fa41f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1fa3400640 1 --2- 172.21.15.131:0/2492665519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa4160240 0x7f1fa41f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.283+0000 7f1fa3400640 1 -- 172.21.15.131:0/2492665519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa40cd940 con 0x7f1fa40d1d70 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.284+0000 7f1fa3400640 1 --2- 172.21.15.131:0/2492665519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fa40d1d70 0x7f1fa40ce9e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1f9402a2b0 tx=0x7f1f940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.284+0000 7f1f9b400640 1 -- 172.21.15.131:0/2492665519 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94041020 con 0x7f1fa40d1d70 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.284+0000 7f1f9b400640 1 -- 172.21.15.131:0/2492665519 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f940306c0 con 0x7f1fa40d1d70 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.284+0000 7f1f9b400640 1 -- 172.21.15.131:0/2492665519 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94037040 con 0x7f1fa40d1d70 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.284+0000 7f1faa400640 1 -- 172.21.15.131:0/2492665519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa40cdc20 con 0x7f1fa40d1d70 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.284+0000 7f1faa400640 1 -- 172.21.15.131:0/2492665519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa40ce1e0 con 0x7f1fa40d1d70 2024-04-28T12:37:50.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.285+0000 7f1faa400640 1 -- 172.21.15.131:0/2492665519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa4069350 con 0x7f1fa40d1d70 2024-04-28T12:37:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.286+0000 7f1f9b400640 1 -- 172.21.15.131:0/2492665519 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f94044020 con 0x7f1fa40d1d70 2024-04-28T12:37:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.287+0000 7f1f9b400640 1 --2- 172.21.15.131:0/2492665519 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1f8406c920 0x7f1f8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.287+0000 7f1f9b400640 1 -- 172.21.15.131:0/2492665519 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1f940ac760 con 0x7f1fa40d1d70 2024-04-28T12:37:50.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.287+0000 7f1fa2a00640 1 --2- 172.21.15.131:0/2492665519 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1f8406c920 0x7f1f8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:50.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.289+0000 7f1fa2a00640 1 --2- 172.21.15.131:0/2492665519 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1f8406c920 0x7f1f8406ede0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40b2fb0 tx=0x7f1f90008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:50.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.292+0000 7f1f9b400640 1 -- 172.21.15.131:0/2492665519 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f9404b030 con 0x7f1fa40d1d70 2024-04-28T12:37:50.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:50.531+0000 7f1faa400640 1 -- 172.21.15.131:0/2492665519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1fa40b3810 con 0x7f1fa40d1d70 2024-04-28T12:37:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.056+0000 7f1f9b400640 1 -- 172.21.15.131:0/2492665519 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f1f9407a7b0 con 0x7f1fa40d1d70 2024-04-28T12:37:51.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.059+0000 7f1faa400640 1 -- 172.21.15.131:0/2492665519 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1f8406c920 msgr2=0x7f1f8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.059+0000 7f1faa400640 1 --2- 172.21.15.131:0/2492665519 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1f8406c920 0x7f1f8406ede0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40b2fb0 tx=0x7f1f90008040 comp rx=0 tx=0).stop 2024-04-28T12:37:51.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.059+0000 7f1faa400640 1 -- 172.21.15.131:0/2492665519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fa40d1d70 msgr2=0x7f1fa40ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.059+0000 7f1faa400640 1 --2- 172.21.15.131:0/2492665519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fa40d1d70 0x7f1fa40ce9e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1f9402a2b0 tx=0x7f1f940046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.059+0000 7f1faa400640 1 -- 172.21.15.131:0/2492665519 shutdown_connections 2024-04-28T12:37:51.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.059+0000 7f1faa400640 1 --2- 172.21.15.131:0/2492665519 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1f8406c920 0x7f1f8406ede0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.059+0000 7f1faa400640 1 --2- 172.21.15.131:0/2492665519 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fa4160240 0x7f1fa41f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.059+0000 7f1faa400640 1 --2- 172.21.15.131:0/2492665519 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fa415cca0 0x7f1fa41f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.059+0000 7f1faa400640 1 --2- 172.21.15.131:0/2492665519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fa40d1d70 0x7f1fa40ce9e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.059+0000 7f1faa400640 1 -- 172.21.15.131:0/2492665519 >> 172.21.15.131:0/2492665519 conn(0x7f1fa40d4e40 msgr2=0x7f1fa406cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:51.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.060+0000 7f1faa400640 1 -- 172.21.15.131:0/2492665519 shutdown_connections 2024-04-28T12:37:51.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.060+0000 7f1faa400640 1 -- 172.21.15.131:0/2492665519 wait complete. 2024-04-28T12:37:51.078 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-28T12:37:51.078 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:37:51.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.284+0000 7f7bd2e00640 1 Processor -- start 2024-04-28T12:37:51.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.285+0000 7f7bd2e00640 1 -- start start 2024-04-28T12:37:51.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.286+0000 7f7bd2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0b5b80 0x7f7bcc0d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.286+0000 7f7bd2e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bcc0d0480 con 0x7f7bcc0d2800 2024-04-28T12:37:51.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.286+0000 7f7bd2e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bcc0d0720 con 0x7f7bcc0d1620 2024-04-28T12:37:51.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.286+0000 7f7bd2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7bcc0d0ce0 con 0x7f7bcc0b5b80 2024-04-28T12:37:51.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.286+0000 7f7bd2400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7bcc0d2800 0x7f7bcc0d2c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33146/0 (socket says 172.21.15.131:33146) 2024-04-28T12:37:51.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.286+0000 7f7bd2400640 1 -- 172.21.15.131:0/759381966 learned_addr learned my addr 172.21.15.131:0/759381966 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:51.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.287+0000 7f7bd1000640 1 --2- 172.21.15.131:0/759381966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0b5b80 0x7f7bcc0d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:51.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.287+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3509059546 0 0) 0x7f7bcc0d0480 con 0x7f7bcc0d2800 2024-04-28T12:37:51.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.287+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bb0003520 con 0x7f7bcc0d2800 2024-04-28T12:37:51.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.287+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3602198892 0 0) 0x7f7bb0003520 con 0x7f7bcc0d2800 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.287+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0b5b80 msgr2=0x7f7bcc0d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.287+0000 7f7bcbe00640 1 --2- 172.21.15.131:0/759381966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0b5b80 0x7f7bcc0d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.287+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 >> v1:172.21.15.157:6789/0 conn(0x7f7bcc0d1620 legacy=0x7f7bcc0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.287+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bcc0d0a00 con 0x7f7bcc0d2800 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.287+0000 7f7bd1000640 1 --2- 172.21.15.131:0/759381966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0b5b80 0x7f7bcc0d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.287+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f7bc4001670 con 0x7f7bcc0d2800 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.287+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 >> v1:172.21.15.131:6789/0 conn(0x7f7bcc0d2800 legacy=0x7f7bcc0d2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.288+0000 7f7bcbe00640 1 --2- 172.21.15.131:0/759381966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0b5b80 0x7f7bb00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.288+0000 7f7bcbe00640 1 --2- 172.21.15.131:0/759381966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb0003f20 0x7f7bb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.288+0000 7f7bcbe00640 1 --2- 172.21.15.131:0/759381966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb0004d40 0x7f7bb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.288+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc0d0ce0 con 0x7f7bb0003f20 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.288+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb000bad0 con 0x7f7bb0004d40 2024-04-28T12:37:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.288+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb000bcd0 con 0x7f7bcc0b5b80 2024-04-28T12:37:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.288+0000 7f7bd1000640 1 --2- 172.21.15.131:0/759381966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb0003f20 0x7f7bb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.288+0000 7f7bd1a00640 1 --2- 172.21.15.131:0/759381966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0b5b80 0x7f7bb00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.289+0000 7f7bd1000640 1 -- 172.21.15.131:0/759381966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0b5b80 msgr2=0x7f7bb00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.289+0000 7f7bd1000640 1 --2- 172.21.15.131:0/759381966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0b5b80 0x7f7bb00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.289+0000 7f7bd1000640 1 -- 172.21.15.131:0/759381966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb0004d40 msgr2=0x7f7bb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.289+0000 7f7bd1000640 1 --2- 172.21.15.131:0/759381966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb0004d40 0x7f7bb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.289+0000 7f7bd1000640 1 -- 172.21.15.131:0/759381966 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb0003520 con 0x7f7bb0003f20 2024-04-28T12:37:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.290+0000 7f7bd1000640 1 --2- 172.21.15.131:0/759381966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb0003f20 0x7f7bb0004800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e350 tx=0x7f7bb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f86901198a05433 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.290+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8009da0 con 0x7f7bb0003f20 2024-04-28T12:37:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.290+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb8003b90 con 0x7f7bb0003f20 2024-04-28T12:37:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.290+0000 7f7bcbe00640 1 -- 172.21.15.131:0/759381966 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8004030 con 0x7f7bb0003f20 2024-04-28T12:37:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.290+0000 7f7bd2e00640 1 -- 172.21.15.131:0/759381966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb0003f20 msgr2=0x7f7bb0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.290+0000 7f7bd2e00640 1 --2- 172.21.15.131:0/759381966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb0003f20 0x7f7bb0004800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e350 tx=0x7f7bb0005830 comp rx=0 tx=0).stop 2024-04-28T12:37:51.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.290+0000 7f7bd2400640 1 -- 172.21.15.131:0/759381966 reap_dead start 2024-04-28T12:37:51.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.291+0000 7f7bd2e00640 1 -- 172.21.15.131:0/759381966 shutdown_connections 2024-04-28T12:37:51.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.291+0000 7f7bd2e00640 1 -- 172.21.15.131:0/759381966 >> 172.21.15.131:0/759381966 conn(0x7f7bcc0d4c30 msgr2=0x7f7bcc0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:51.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.291+0000 7f7bd2e00640 1 -- 172.21.15.131:0/759381966 shutdown_connections 2024-04-28T12:37:51.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.291+0000 7f7bd2e00640 1 -- 172.21.15.131:0/759381966 wait complete. 2024-04-28T12:37:51.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.292+0000 7f7bd2e00640 1 Processor -- start 2024-04-28T12:37:51.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.293+0000 7f7bd2e00640 1 -- start start 2024-04-28T12:37:51.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.293+0000 7f7bd2e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb0003f20 0x7f7bcc1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.293+0000 7f7bd2e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bcc1f8bc0 0x7f7bcc14ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.293+0000 7f7bd2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0d2c50 0x7f7bcc0d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.294+0000 7f7bd2e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc0d58d0 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.294+0000 7f7bd2e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc0d5750 con 0x7f7bb0003f20 2024-04-28T12:37:51.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.294+0000 7f7bd2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7bcc0d5a50 con 0x7f7bcc0d2c50 2024-04-28T12:37:51.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.294+0000 7f7bd1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bcc1f8bc0 0x7f7bcc14ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:51.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.294+0000 7f7bd1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bcc1f8bc0 0x7f7bcc14ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45042/0 (socket says 172.21.15.131:45042) 2024-04-28T12:37:51.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.294+0000 7f7bd1000640 1 -- 172.21.15.131:0/1143480390 learned_addr learned my addr 172.21.15.131:0/1143480390 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:51.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.294+0000 7f7bd1a00640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb0003f20 0x7f7bcc1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:51.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.294+0000 7f7bd1000640 1 -- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0d2c50 msgr2=0x7f7bcc0d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.294+0000 7f7bd1000640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0d2c50 0x7f7bcc0d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.294+0000 7f7bd1000640 1 -- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb0003f20 msgr2=0x7f7bcc1515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.294+0000 7f7bd1000640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb0003f20 0x7f7bcc1515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.294+0000 7f7bd1000640 1 -- 172.21.15.131:0/1143480390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bcc151cb0 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.295+0000 7f7bd1000640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bcc1f8bc0 0x7f7bcc14ec20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7bb802b040 tx=0x7f7bb8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:51.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.295+0000 7f7bc9600640 1 -- 172.21.15.131:0/1143480390 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8009da0 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.295+0000 7f7bc9600640 1 -- 172.21.15.131:0/1143480390 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb8003990 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.295+0000 7f7bc9600640 1 -- 172.21.15.131:0/1143480390 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb802f6a0 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.295+0000 7f7bd2e00640 1 -- 172.21.15.131:0/1143480390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bcc151f90 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.295+0000 7f7bd2e00640 1 -- 172.21.15.131:0/1143480390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bcc1f9040 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.296+0000 7f7bd2e00640 1 -- 172.21.15.131:0/1143480390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bcc0d3610 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.297+0000 7f7bc9600640 1 -- 172.21.15.131:0/1143480390 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bb802f840 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.298+0000 7f7bc9600640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7bac06c8f0 0x7f7bac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.298+0000 7f7bc9600640 1 -- 172.21.15.131:0/1143480390 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7bb80aceb0 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.299+0000 7f7bd1a00640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7bac06c8f0 0x7f7bac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:51.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.300+0000 7f7bd1a00640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7bac06c8f0 0x7f7bac06edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc005500 tx=0x7f7bbc007f10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:51.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.303+0000 7f7bc9600640 1 -- 172.21.15.131:0/1143480390 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7bb8033020 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.544+0000 7f7bd2e00640 1 -- 172.21.15.131:0/1143480390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bcc0b3d50 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.545+0000 7f7bc9600640 1 -- 172.21.15.131:0/1143480390 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5191 (secure 0 0 0) 0x7f7bb807af90 con 0x7f7bcc1f8bc0 2024-04-28T12:37:51.542 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:51.543 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1064054651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1064054651},{"type":"v1","addr":"172.21.15.131:6834","nonce":1064054651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/398655652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":398655652},{"type":"v1","addr":"172.21.15.131:6835","nonce":398655652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2619661531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2619661531},{"type":"v1","addr":"172.21.15.157:6837","nonce":2619661531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:37:22.236351+0000","modified":"2024-04-28T12:37:50.533095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1937780695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1937780695},{"type":"v1","addr":"172.21.15.157:6835","nonce":1937780695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34160,"qdb_cluster":[34160]},"id":1}]} 2024-04-28T12:37:51.543 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-04-28T12:37:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.548+0000 7f7bd2e00640 1 -- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7bac06c8f0 msgr2=0x7f7bac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.548+0000 7f7bd2e00640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7bac06c8f0 0x7f7bac06edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc005500 tx=0x7f7bbc007f10 comp rx=0 tx=0).stop 2024-04-28T12:37:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.548+0000 7f7bd2e00640 1 -- 172.21.15.131:0/1143480390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bcc1f8bc0 msgr2=0x7f7bcc14ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.548+0000 7f7bd2e00640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bcc1f8bc0 0x7f7bcc14ec20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7bb802b040 tx=0x7f7bb8009750 comp rx=0 tx=0).stop 2024-04-28T12:37:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.549+0000 7f7bd2e00640 1 -- 172.21.15.131:0/1143480390 shutdown_connections 2024-04-28T12:37:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.549+0000 7f7bd2e00640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7bac06c8f0 0x7f7bac06edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.549+0000 7f7bd2e00640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bcc0d2c50 0x7f7bcc0d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.549+0000 7f7bd2e00640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bcc1f8bc0 0x7f7bcc14ec20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.549+0000 7f7bd2e00640 1 --2- 172.21.15.131:0/1143480390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bb0003f20 0x7f7bcc1515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.549+0000 7f7bd2e00640 1 -- 172.21.15.131:0/1143480390 >> 172.21.15.131:0/1143480390 conn(0x7f7bcc0d4c30 msgr2=0x7f7bcc0d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.549+0000 7f7bd2e00640 1 -- 172.21.15.131:0/1143480390 shutdown_connections 2024-04-28T12:37:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.549+0000 7f7bd2e00640 1 -- 172.21.15.131:0/1143480390 wait complete. 2024-04-28T12:37:51.564 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:51.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.770+0000 7f2b4d600640 1 Processor -- start 2024-04-28T12:37:51.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.771+0000 7f2b4d600640 1 -- start start 2024-04-28T12:37:51.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.772+0000 7f2b4d600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b48145590 0x7f2b480ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.772+0000 7f2b4d600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2b4806f730 0x7f2b48071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.772+0000 7f2b4d600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b480cf060 con 0x7f2b480b3490 2024-04-28T12:37:51.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.772+0000 7f2b4d600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2b480d0100 con 0x7f2b4806f730 2024-04-28T12:37:51.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.772+0000 7f2b4d600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2b480cf340 con 0x7f2b48145590 2024-04-28T12:37:51.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.772+0000 7f2b47400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2b4806f730 0x7f2b48071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:51.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.772+0000 7f2b46000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2b480b3490 0x7f2b4806f020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33150/0 (socket says 172.21.15.131:33150) 2024-04-28T12:37:51.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.772+0000 7f2b46000640 1 -- 172.21.15.131:0/2144838703 learned_addr learned my addr 172.21.15.131:0/2144838703 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:51.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.773+0000 7f2b46a00640 1 --2- 172.21.15.131:0/2144838703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b48145590 0x7f2b480ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:51.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.773+0000 7f2b45600640 1 -- 172.21.15.131:0/2144838703 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3144748384 0 0) 0x7f2b480cf060 con 0x7f2b480b3490 2024-04-28T12:37:51.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.773+0000 7f2b47400640 1 -- 172.21.15.131:0/2144838703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b48145590 msgr2=0x7f2b480ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:51.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.773+0000 7f2b47400640 1 --2- 172.21.15.131:0/2144838703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b48145590 0x7f2b480ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.773+0000 7f2b47400640 1 -- 172.21.15.131:0/2144838703 >> v1:172.21.15.131:6789/0 conn(0x7f2b480b3490 legacy=0x7f2b4806f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.773+0000 7f2b47400640 1 -- 172.21.15.131:0/2144838703 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b480cf620 con 0x7f2b4806f730 2024-04-28T12:37:51.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.774+0000 7f2b47400640 1 --2- 172.21.15.131:0/2144838703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b4806f730 0x7f2b48071eb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009d90 tx=0x7f2b4002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0be0bbb76f36b3f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:51.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.774+0000 7f2b45600640 1 -- 172.21.15.131:0/2144838703 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40007740 con 0x7f2b4806f730 2024-04-28T12:37:51.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.775+0000 7f2b45600640 1 -- 172.21.15.131:0/2144838703 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b400078e0 con 0x7f2b4806f730 2024-04-28T12:37:51.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.775+0000 7f2b45600640 1 -- 172.21.15.131:0/2144838703 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40007cb0 con 0x7f2b4806f730 2024-04-28T12:37:51.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.775+0000 7f2b4d600640 1 -- 172.21.15.131:0/2144838703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b4806f730 msgr2=0x7f2b48071eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.775+0000 7f2b4d600640 1 --2- 172.21.15.131:0/2144838703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b4806f730 0x7f2b48071eb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009d90 tx=0x7f2b4002a660 comp rx=0 tx=0).stop 2024-04-28T12:37:51.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.775+0000 7f2b4d600640 1 -- 172.21.15.131:0/2144838703 shutdown_connections 2024-04-28T12:37:51.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.775+0000 7f2b4d600640 1 --2- 172.21.15.131:0/2144838703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b4806f730 0x7f2b48071eb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.775+0000 7f2b4d600640 1 --2- 172.21.15.131:0/2144838703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b48145590 0x7f2b480ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.775+0000 7f2b4d600640 1 -- 172.21.15.131:0/2144838703 >> 172.21.15.131:0/2144838703 conn(0x7f2b480b2900 msgr2=0x7f2b480b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:51.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.775+0000 7f2b4d600640 1 -- 172.21.15.131:0/2144838703 shutdown_connections 2024-04-28T12:37:51.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.775+0000 7f2b4d600640 1 -- 172.21.15.131:0/2144838703 wait complete. 2024-04-28T12:37:51.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.777+0000 7f2b4d600640 1 Processor -- start 2024-04-28T12:37:51.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.777+0000 7f2b4d600640 1 -- start start 2024-04-28T12:37:51.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.778+0000 7f2b4d600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b4806f730 0x7f2b480cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.778+0000 7f2b4d600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b480b3490 0x7f2b481e8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.778+0000 7f2b4d600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b48145590 0x7f2b481e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.778+0000 7f2b4d600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2b48145970 con 0x7f2b480b3490 2024-04-28T12:37:51.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.778+0000 7f2b4d600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2b480cfbe0 con 0x7f2b48145590 2024-04-28T12:37:51.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.778+0000 7f2b4d600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2b480cecc0 con 0x7f2b4806f730 2024-04-28T12:37:51.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.778+0000 7f2b46000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b480b3490 0x7f2b481e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:51.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.779+0000 7f2b46000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b480b3490 0x7f2b481e8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45054/0 (socket says 172.21.15.131:45054) 2024-04-28T12:37:51.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.779+0000 7f2b46000640 1 -- 172.21.15.131:0/3609144690 learned_addr learned my addr 172.21.15.131:0/3609144690 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:51.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.779+0000 7f2b47400640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b48145590 0x7f2b481e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:51.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.779+0000 7f2b46000640 1 -- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b4806f730 msgr2=0x7f2b480cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.779+0000 7f2b46000640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b4806f730 0x7f2b480cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.779+0000 7f2b46000640 1 -- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b48145590 msgr2=0x7f2b481e8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.779+0000 7f2b46000640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b48145590 0x7f2b481e8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.779+0000 7f2b46000640 1 -- 172.21.15.131:0/3609144690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4815c5b0 con 0x7f2b480b3490 2024-04-28T12:37:51.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.779+0000 7f2b46000640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b480b3490 0x7f2b481e8540 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800c4e0 tx=0x7f2b3800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:51.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.779+0000 7f2b3ea00640 1 -- 172.21.15.131:0/3609144690 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38018070 con 0x7f2b480b3490 2024-04-28T12:37:51.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.780+0000 7f2b3ea00640 1 -- 172.21.15.131:0/3609144690 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b38007500 con 0x7f2b480b3490 2024-04-28T12:37:51.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.780+0000 7f2b3ea00640 1 -- 172.21.15.131:0/3609144690 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38004a20 con 0x7f2b480b3490 2024-04-28T12:37:51.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.780+0000 7f2b4d600640 1 -- 172.21.15.131:0/3609144690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4815c890 con 0x7f2b480b3490 2024-04-28T12:37:51.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.780+0000 7f2b4d600640 1 -- 172.21.15.131:0/3609144690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4815ce50 con 0x7f2b480b3490 2024-04-28T12:37:51.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.781+0000 7f2b4d600640 1 -- 172.21.15.131:0/3609144690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b48069350 con 0x7f2b480b3490 2024-04-28T12:37:51.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.782+0000 7f2b3ea00640 1 -- 172.21.15.131:0/3609144690 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b38004bc0 con 0x7f2b480b3490 2024-04-28T12:37:51.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.783+0000 7f2b3ea00640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2b2406c8f0 0x7f2b2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:51.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.783+0000 7f2b3ea00640 1 -- 172.21.15.131:0/3609144690 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2b3808b2a0 con 0x7f2b480b3490 2024-04-28T12:37:51.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.784+0000 7f2b46a00640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2b2406c8f0 0x7f2b2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:51.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.785+0000 7f2b46a00640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2b2406c8f0 0x7f2b2406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b30006fd0 tx=0x7f2b30008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:51.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.788+0000 7f2b3ea00640 1 -- 172.21.15.131:0/3609144690 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2b38059440 con 0x7f2b480b3490 2024-04-28T12:37:51.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.988+0000 7f2b4d600640 1 -- 172.21.15.131:0/3609144690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b48142c60 con 0x7f2b480b3490 2024-04-28T12:37:51.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.991+0000 7f2b3ea00640 1 -- 172.21.15.131:0/3609144690 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7f2b38058d40 con 0x7f2b480b3490 2024-04-28T12:37:51.988 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:51.988 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.994+0000 7f2b4d600640 1 -- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2b2406c8f0 msgr2=0x7f2b2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.994+0000 7f2b4d600640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2b2406c8f0 0x7f2b2406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b30006fd0 tx=0x7f2b30008040 comp rx=0 tx=0).stop 2024-04-28T12:37:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.994+0000 7f2b4d600640 1 -- 172.21.15.131:0/3609144690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b480b3490 msgr2=0x7f2b481e8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.994+0000 7f2b4d600640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b480b3490 0x7f2b481e8540 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800c4e0 tx=0x7f2b3800c9a0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.994+0000 7f2b4d600640 1 -- 172.21.15.131:0/3609144690 shutdown_connections 2024-04-28T12:37:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.994+0000 7f2b4d600640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2b2406c8f0 0x7f2b2406edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.994+0000 7f2b4d600640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b48145590 0x7f2b481e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.994+0000 7f2b4d600640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b480b3490 0x7f2b481e8540 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.994+0000 7f2b4d600640 1 --2- 172.21.15.131:0/3609144690 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b4806f730 0x7f2b480cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.994+0000 7f2b4d600640 1 -- 172.21.15.131:0/3609144690 >> 172.21.15.131:0/3609144690 conn(0x7f2b480b2900 msgr2=0x7f2b480b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.994+0000 7f2b4d600640 1 -- 172.21.15.131:0/3609144690 shutdown_connections 2024-04-28T12:37:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:51.994+0000 7f2b4d600640 1 -- 172.21.15.131:0/3609144690 wait complete. 2024-04-28T12:37:52.011 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-28T12:37:52.011 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:37:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.213+0000 7efe6ca00640 1 Processor -- start 2024-04-28T12:37:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.213+0000 7efe6ca00640 1 -- start start 2024-04-28T12:37:52.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.215+0000 7efe6ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe680cd180 0x7efe68072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.215+0000 7efe6ca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efe6815dc50 0x7efe680cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.215+0000 7efe6ca00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe680cd560 con 0x7efe680ce2b0 2024-04-28T12:37:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.215+0000 7efe6ca00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7efe68072d00 con 0x7efe6815dc50 2024-04-28T12:37:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.215+0000 7efe6ca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efe68072fe0 con 0x7efe680cd180 2024-04-28T12:37:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.215+0000 7efe5fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efe6815dc50 0x7efe680cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.215+0000 7efe5fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efe6815dc50 0x7efe680cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45068/0 (socket says 172.21.15.131:45068) 2024-04-28T12:37:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.215+0000 7efe5fe00640 1 -- 172.21.15.131:0/16741079 learned_addr learned my addr 172.21.15.131:0/16741079 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.216+0000 7efe67400640 1 --2- 172.21.15.131:0/16741079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe680cd180 0x7efe68072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.216+0000 7efe5fe00640 1 -- 172.21.15.131:0/16741079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe680cd180 msgr2=0x7efe68072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.216+0000 7efe5fe00640 1 --2- 172.21.15.131:0/16741079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe680cd180 0x7efe68072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.216+0000 7efe5fe00640 1 -- 172.21.15.131:0/16741079 >> v1:172.21.15.157:6789/0 conn(0x7efe680ce2b0 legacy=0x7efe681607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.216+0000 7efe5fe00640 1 -- 172.21.15.131:0/16741079 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe680732c0 con 0x7efe6815dc50 2024-04-28T12:37:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.217+0000 7efe5fe00640 1 --2- 172.21.15.131:0/16741079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe6815dc50 0x7efe680cdd70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efe50009870 tx=0x7efe5002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c4555dee73935a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.217+0000 7efe66a00640 1 -- 172.21.15.131:0/16741079 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50037070 con 0x7efe6815dc50 2024-04-28T12:37:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.217+0000 7efe66a00640 1 -- 172.21.15.131:0/16741079 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe50032440 con 0x7efe6815dc50 2024-04-28T12:37:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.217+0000 7efe66a00640 1 -- 172.21.15.131:0/16741079 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50032810 con 0x7efe6815dc50 2024-04-28T12:37:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.218+0000 7efe6ca00640 1 -- 172.21.15.131:0/16741079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe6815dc50 msgr2=0x7efe680cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.218+0000 7efe6ca00640 1 --2- 172.21.15.131:0/16741079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe6815dc50 0x7efe680cdd70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efe50009870 tx=0x7efe5002a210 comp rx=0 tx=0).stop 2024-04-28T12:37:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.218+0000 7efe6ca00640 1 -- 172.21.15.131:0/16741079 shutdown_connections 2024-04-28T12:37:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.218+0000 7efe6ca00640 1 --2- 172.21.15.131:0/16741079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe6815dc50 0x7efe680cdd70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.218+0000 7efe6ca00640 1 --2- 172.21.15.131:0/16741079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe680cd180 0x7efe68072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.218+0000 7efe6ca00640 1 -- 172.21.15.131:0/16741079 >> 172.21.15.131:0/16741079 conn(0x7efe68068090 msgr2=0x7efe68068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.218+0000 7efe6ca00640 1 -- 172.21.15.131:0/16741079 shutdown_connections 2024-04-28T12:37:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.218+0000 7efe6ca00640 1 -- 172.21.15.131:0/16741079 wait complete. 2024-04-28T12:37:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.220+0000 7efe6ca00640 1 Processor -- start 2024-04-28T12:37:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.220+0000 7efe6ca00640 1 -- start start 2024-04-28T12:37:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe6ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe680cd180 0x7efe681e6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe6ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe680ce2b0 0x7efe6815c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe6ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe6815dc50 0x7efe681f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe6ca00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efe6806ead0 con 0x7efe680ce2b0 2024-04-28T12:37:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe6ca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efe68073da0 con 0x7efe680cd180 2024-04-28T12:37:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe6ca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efe6815e0d0 con 0x7efe6815dc50 2024-04-28T12:37:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe5fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe680ce2b0 0x7efe6815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe5fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe680ce2b0 0x7efe6815c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45080/0 (socket says 172.21.15.131:45080) 2024-04-28T12:37:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe5fe00640 1 -- 172.21.15.131:0/1746361609 learned_addr learned my addr 172.21.15.131:0/1746361609 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe67e00640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe6815dc50 0x7efe681f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe67400640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe680cd180 0x7efe681e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe5fe00640 1 -- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe6815dc50 msgr2=0x7efe681f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe5fe00640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe6815dc50 0x7efe681f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe5fe00640 1 -- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe680cd180 msgr2=0x7efe681e6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe5fe00640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe680cd180 0x7efe681e6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe5fe00640 1 -- 172.21.15.131:0/1746361609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe681e6bf0 con 0x7efe680ce2b0 2024-04-28T12:37:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.221+0000 7efe67e00640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe6815dc50 0x7efe681f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.222+0000 7efe5fe00640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe680ce2b0 0x7efe6815c680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efe50009870 tx=0x7efe500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.222+0000 7efe5f400640 1 -- 172.21.15.131:0/1746361609 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50041020 con 0x7efe680ce2b0 2024-04-28T12:37:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.222+0000 7efe5f400640 1 -- 172.21.15.131:0/1746361609 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe500306c0 con 0x7efe680ce2b0 2024-04-28T12:37:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.222+0000 7efe5f400640 1 -- 172.21.15.131:0/1746361609 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50037040 con 0x7efe680ce2b0 2024-04-28T12:37:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.222+0000 7efe6ca00640 1 -- 172.21.15.131:0/1746361609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe681e6ed0 con 0x7efe680ce2b0 2024-04-28T12:37:52.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.222+0000 7efe6ca00640 1 -- 172.21.15.131:0/1746361609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe681e7490 con 0x7efe680ce2b0 2024-04-28T12:37:52.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.223+0000 7efe6ca00640 1 -- 172.21.15.131:0/1746361609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe68069350 con 0x7efe680ce2b0 2024-04-28T12:37:52.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.224+0000 7efe5f400640 1 -- 172.21.15.131:0/1746361609 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe50044020 con 0x7efe680ce2b0 2024-04-28T12:37:52.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.225+0000 7efe5f400640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efe4406c8f0 0x7efe4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:52.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.225+0000 7efe5f400640 1 -- 172.21.15.131:0/1746361609 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7efe500ac730 con 0x7efe680ce2b0 2024-04-28T12:37:52.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.226+0000 7efe67400640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efe4406c8f0 0x7efe4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:52.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.227+0000 7efe67400640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efe4406c8f0 0x7efe4406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efe58006fd0 tx=0x7efe58008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.230+0000 7efe5f400640 1 -- 172.21.15.131:0/1746361609 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe5004b030 con 0x7efe680ce2b0 2024-04-28T12:37:52.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.427+0000 7efe6ca00640 1 -- 172.21.15.131:0/1746361609 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe681f4450 con 0x7efe4406c8f0 2024-04-28T12:37:52.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.458+0000 7efe5f400640 1 -- 172.21.15.131:0/1746361609 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332538 (secure 0 0 0) 0x7efe681f4450 con 0x7efe4406c8f0 2024-04-28T12:37:52.460 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:52.461 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:37:52.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.464+0000 7efe6ca00640 1 -- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efe4406c8f0 msgr2=0x7efe4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:52.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.464+0000 7efe6ca00640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efe4406c8f0 0x7efe4406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efe58006fd0 tx=0x7efe58008040 comp rx=0 tx=0).stop 2024-04-28T12:37:52.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.464+0000 7efe6ca00640 1 -- 172.21.15.131:0/1746361609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe680ce2b0 msgr2=0x7efe6815c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:52.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.464+0000 7efe6ca00640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe680ce2b0 0x7efe6815c680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efe50009870 tx=0x7efe500046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.464+0000 7efe6ca00640 1 -- 172.21.15.131:0/1746361609 shutdown_connections 2024-04-28T12:37:52.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.464+0000 7efe6ca00640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efe4406c8f0 0x7efe4406edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.464+0000 7efe6ca00640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efe6815dc50 0x7efe681f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.464+0000 7efe6ca00640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe680ce2b0 0x7efe6815c680 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.464+0000 7efe6ca00640 1 --2- 172.21.15.131:0/1746361609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe680cd180 0x7efe681e6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.464+0000 7efe6ca00640 1 -- 172.21.15.131:0/1746361609 >> 172.21.15.131:0/1746361609 conn(0x7efe68068090 msgr2=0x7efe680b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:52.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.465+0000 7efe6ca00640 1 -- 172.21.15.131:0/1746361609 shutdown_connections 2024-04-28T12:37:52.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.465+0000 7efe6ca00640 1 -- 172.21.15.131:0/1746361609 wait complete. 2024-04-28T12:37:52.479 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-28T12:37:50.924285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231864,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742856,"statfs":{"total":773094113280,"available":772856684544,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005164"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980293+0000","last_change":"2024-04-28T12:37:21.812889+0000","last_active":"2024-04-28T12:37:27.980293+0000","last_peered":"2024-04-28T12:37:27.980293+0000","last_clean":"2024-04-28T12:37:27.980293+0000","last_became_active":"2024-04-28T12:37:21.812593+0000","last_became_peered":"2024-04-28T12:37:21.812593+0000","last_unstale":"2024-04-28T12:37:27.980293+0000","last_undegraded":"2024-04-28T12:37:27.980293+0000","last_fullsized":"2024-04-28T12:37:27.980293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:33.616030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978052+0000","last_change":"2024-04-28T12:37:22.794799+0000","last_active":"2024-04-28T12:37:27.978052+0000","last_peered":"2024-04-28T12:37:27.978052+0000","last_clean":"2024-04-28T12:37:27.978052+0000","last_became_active":"2024-04-28T12:37:22.794480+0000","last_became_peered":"2024-04-28T12:37:22.794480+0000","last_unstale":"2024-04-28T12:37:27.978052+0000","last_undegraded":"2024-04-28T12:37:27.978052+0000","last_fullsized":"2024-04-28T12:37:27.978052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:39.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976789+0000","last_change":"2024-04-28T12:37:26.883025+0000","last_active":"2024-04-28T12:37:27.976789+0000","last_peered":"2024-04-28T12:37:27.976789+0000","last_clean":"2024-04-28T12:37:27.976789+0000","last_became_active":"2024-04-28T12:37:26.882577+0000","last_became_peered":"2024-04-28T12:37:26.882577+0000","last_unstale":"2024-04-28T12:37:27.976789+0000","last_undegraded":"2024-04-28T12:37:27.976789+0000","last_fullsized":"2024-04-28T12:37:27.976789+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:09.634939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978274+0000","last_change":"2024-04-28T12:37:21.796325+0000","last_active":"2024-04-28T12:37:27.978274+0000","last_peered":"2024-04-28T12:37:27.978274+0000","last_clean":"2024-04-28T12:37:27.978274+0000","last_became_active":"2024-04-28T12:37:21.795878+0000","last_became_peered":"2024-04-28T12:37:21.795878+0000","last_unstale":"2024-04-28T12:37:27.978274+0000","last_undegraded":"2024-04-28T12:37:27.978274+0000","last_fullsized":"2024-04-28T12:37:27.978274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:55:28.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979406+0000","last_change":"2024-04-28T12:37:22.821287+0000","last_active":"2024-04-28T12:37:27.979406+0000","last_peered":"2024-04-28T12:37:27.979406+0000","last_clean":"2024-04-28T12:37:27.979406+0000","last_became_active":"2024-04-28T12:37:22.820999+0000","last_became_peered":"2024-04-28T12:37:22.820999+0000","last_unstale":"2024-04-28T12:37:27.979406+0000","last_undegraded":"2024-04-28T12:37:27.979406+0000","last_fullsized":"2024-04-28T12:37:27.979406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843680+0000","last_change":"2024-04-28T12:37:26.860456+0000","last_active":"2024-04-28T12:37:27.843680+0000","last_peered":"2024-04-28T12:37:27.843680+0000","last_clean":"2024-04-28T12:37:27.843680+0000","last_became_active":"2024-04-28T12:37:26.859360+0000","last_became_peered":"2024-04-28T12:37:26.859360+0000","last_unstale":"2024-04-28T12:37:27.843680+0000","last_undegraded":"2024-04-28T12:37:27.843680+0000","last_fullsized":"2024-04-28T12:37:27.843680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:17.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982655+0000","last_change":"2024-04-28T12:37:21.817405+0000","last_active":"2024-04-28T12:37:27.982655+0000","last_peered":"2024-04-28T12:37:27.982655+0000","last_clean":"2024-04-28T12:37:27.982655+0000","last_became_active":"2024-04-28T12:37:21.816935+0000","last_became_peered":"2024-04-28T12:37:21.816935+0000","last_unstale":"2024-04-28T12:37:27.982655+0000","last_undegraded":"2024-04-28T12:37:27.982655+0000","last_fullsized":"2024-04-28T12:37:27.982655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:32.253270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978974+0000","last_change":"2024-04-28T12:37:22.809893+0000","last_active":"2024-04-28T12:37:27.978974+0000","last_peered":"2024-04-28T12:37:27.978974+0000","last_clean":"2024-04-28T12:37:27.978974+0000","last_became_active":"2024-04-28T12:37:22.808912+0000","last_became_peered":"2024-04-28T12:37:22.808912+0000","last_unstale":"2024-04-28T12:37:27.978974+0000","last_undegraded":"2024-04-28T12:37:27.978974+0000","last_fullsized":"2024-04-28T12:37:27.978974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:39.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843302+0000","last_change":"2024-04-28T12:37:26.870945+0000","last_active":"2024-04-28T12:37:27.843302+0000","last_peered":"2024-04-28T12:37:27.843302+0000","last_clean":"2024-04-28T12:37:27.843302+0000","last_became_active":"2024-04-28T12:37:26.867901+0000","last_became_peered":"2024-04-28T12:37:26.867901+0000","last_unstale":"2024-04-28T12:37:27.843302+0000","last_undegraded":"2024-04-28T12:37:27.843302+0000","last_fullsized":"2024-04-28T12:37:27.843302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:50.835488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981355+0000","last_change":"2024-04-28T12:37:21.816630+0000","last_active":"2024-04-28T12:37:27.981355+0000","last_peered":"2024-04-28T12:37:27.981355+0000","last_clean":"2024-04-28T12:37:27.981355+0000","last_became_active":"2024-04-28T12:37:21.815735+0000","last_became_peered":"2024-04-28T12:37:21.815735+0000","last_unstale":"2024-04-28T12:37:27.981355+0000","last_undegraded":"2024-04-28T12:37:27.981355+0000","last_fullsized":"2024-04-28T12:37:27.981355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:46:26.715438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979607+0000","last_change":"2024-04-28T12:37:22.810382+0000","last_active":"2024-04-28T12:37:27.979607+0000","last_peered":"2024-04-28T12:37:27.979607+0000","last_clean":"2024-04-28T12:37:27.979607+0000","last_became_active":"2024-04-28T12:37:22.809175+0000","last_became_peered":"2024-04-28T12:37:22.809175+0000","last_unstale":"2024-04-28T12:37:27.979607+0000","last_undegraded":"2024-04-28T12:37:27.979607+0000","last_fullsized":"2024-04-28T12:37:27.979607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:45.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844410+0000","last_change":"2024-04-28T12:37:26.864046+0000","last_active":"2024-04-28T12:37:27.844410+0000","last_peered":"2024-04-28T12:37:27.844410+0000","last_clean":"2024-04-28T12:37:27.844410+0000","last_became_active":"2024-04-28T12:37:26.863585+0000","last_became_peered":"2024-04-28T12:37:26.863585+0000","last_unstale":"2024-04-28T12:37:27.844410+0000","last_undegraded":"2024-04-28T12:37:27.844410+0000","last_fullsized":"2024-04-28T12:37:27.844410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:26:03.360606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842825+0000","last_change":"2024-04-28T12:37:22.794854+0000","last_active":"2024-04-28T12:37:27.842825+0000","last_peered":"2024-04-28T12:37:27.842825+0000","last_clean":"2024-04-28T12:37:27.842825+0000","last_became_active":"2024-04-28T12:37:22.794522+0000","last_became_peered":"2024-04-28T12:37:22.794522+0000","last_unstale":"2024-04-28T12:37:27.842825+0000","last_undegraded":"2024-04-28T12:37:27.842825+0000","last_fullsized":"2024-04-28T12:37:27.842825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981814+0000","last_change":"2024-04-28T12:37:21.814514+0000","last_active":"2024-04-28T12:37:27.981814+0000","last_peered":"2024-04-28T12:37:27.981814+0000","last_clean":"2024-04-28T12:37:27.981814+0000","last_became_active":"2024-04-28T12:37:21.814226+0000","last_became_peered":"2024-04-28T12:37:21.814226+0000","last_unstale":"2024-04-28T12:37:27.981814+0000","last_undegraded":"2024-04-28T12:37:27.981814+0000","last_fullsized":"2024-04-28T12:37:27.981814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.846440+0000","last_change":"2024-04-28T12:37:26.869683+0000","last_active":"2024-04-28T12:37:27.846440+0000","last_peered":"2024-04-28T12:37:27.846440+0000","last_clean":"2024-04-28T12:37:27.846440+0000","last_became_active":"2024-04-28T12:37:26.869160+0000","last_became_peered":"2024-04-28T12:37:26.869160+0000","last_unstale":"2024-04-28T12:37:27.846440+0000","last_undegraded":"2024-04-28T12:37:27.846440+0000","last_fullsized":"2024-04-28T12:37:27.846440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977688+0000","last_change":"2024-04-28T12:37:22.797734+0000","last_active":"2024-04-28T12:37:27.977688+0000","last_peered":"2024-04-28T12:37:27.977688+0000","last_clean":"2024-04-28T12:37:27.977688+0000","last_became_active":"2024-04-28T12:37:22.797443+0000","last_became_peered":"2024-04-28T12:37:22.797443+0000","last_unstale":"2024-04-28T12:37:27.977688+0000","last_undegraded":"2024-04-28T12:37:27.977688+0000","last_fullsized":"2024-04-28T12:37:27.977688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978047+0000","last_change":"2024-04-28T12:37:21.798038+0000","last_active":"2024-04-28T12:37:27.978047+0000","last_peered":"2024-04-28T12:37:27.978047+0000","last_clean":"2024-04-28T12:37:27.978047+0000","last_became_active":"2024-04-28T12:37:21.797779+0000","last_became_peered":"2024-04-28T12:37:21.797779+0000","last_unstale":"2024-04-28T12:37:27.978047+0000","last_undegraded":"2024-04-28T12:37:27.978047+0000","last_fullsized":"2024-04-28T12:37:27.978047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:32.253270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845656+0000","last_change":"2024-04-28T12:37:26.869060+0000","last_active":"2024-04-28T12:37:27.845656+0000","last_peered":"2024-04-28T12:37:27.845656+0000","last_clean":"2024-04-28T12:37:27.845656+0000","last_became_active":"2024-04-28T12:37:26.867405+0000","last_became_peered":"2024-04-28T12:37:26.867405+0000","last_unstale":"2024-04-28T12:37:27.845656+0000","last_undegraded":"2024-04-28T12:37:27.845656+0000","last_fullsized":"2024-04-28T12:37:27.845656+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:29:29.617319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977155+0000","last_change":"2024-04-28T12:37:22.789718+0000","last_active":"2024-04-28T12:37:27.977155+0000","last_peered":"2024-04-28T12:37:27.977155+0000","last_clean":"2024-04-28T12:37:27.977155+0000","last_became_active":"2024-04-28T12:37:22.786954+0000","last_became_peered":"2024-04-28T12:37:22.786954+0000","last_unstale":"2024-04-28T12:37:27.977155+0000","last_undegraded":"2024-04-28T12:37:27.977155+0000","last_fullsized":"2024-04-28T12:37:27.977155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.975988+0000","last_change":"2024-04-28T12:37:21.804262+0000","last_active":"2024-04-28T12:37:27.975988+0000","last_peered":"2024-04-28T12:37:27.975988+0000","last_clean":"2024-04-28T12:37:27.975988+0000","last_became_active":"2024-04-28T12:37:21.803851+0000","last_became_peered":"2024-04-28T12:37:21.803851+0000","last_unstale":"2024-04-28T12:37:27.975988+0000","last_undegraded":"2024-04-28T12:37:27.975988+0000","last_fullsized":"2024-04-28T12:37:27.975988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:39.334819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978307+0000","last_change":"2024-04-28T12:37:26.884595+0000","last_active":"2024-04-28T12:37:27.978307+0000","last_peered":"2024-04-28T12:37:27.978307+0000","last_clean":"2024-04-28T12:37:27.978307+0000","last_became_active":"2024-04-28T12:37:26.883799+0000","last_became_peered":"2024-04-28T12:37:26.883799+0000","last_unstale":"2024-04-28T12:37:27.978307+0000","last_undegraded":"2024-04-28T12:37:27.978307+0000","last_fullsized":"2024-04-28T12:37:27.978307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:28.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844830+0000","last_change":"2024-04-28T12:37:22.778971+0000","last_active":"2024-04-28T12:37:27.844830+0000","last_peered":"2024-04-28T12:37:27.844830+0000","last_clean":"2024-04-28T12:37:27.844830+0000","last_became_active":"2024-04-28T12:37:22.778674+0000","last_became_peered":"2024-04-28T12:37:22.778674+0000","last_unstale":"2024-04-28T12:37:27.844830+0000","last_undegraded":"2024-04-28T12:37:27.844830+0000","last_fullsized":"2024-04-28T12:37:27.844830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:54.666754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979280+0000","last_change":"2024-04-28T12:37:21.816284+0000","last_active":"2024-04-28T12:37:27.979280+0000","last_peered":"2024-04-28T12:37:27.979280+0000","last_clean":"2024-04-28T12:37:27.979280+0000","last_became_active":"2024-04-28T12:37:21.815140+0000","last_became_peered":"2024-04-28T12:37:21.815140+0000","last_unstale":"2024-04-28T12:37:27.979280+0000","last_undegraded":"2024-04-28T12:37:27.979280+0000","last_fullsized":"2024-04-28T12:37:27.979280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:53.650061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.846043+0000","last_change":"2024-04-28T12:37:26.868136+0000","last_active":"2024-04-28T12:37:27.846043+0000","last_peered":"2024-04-28T12:37:27.846043+0000","last_clean":"2024-04-28T12:37:27.846043+0000","last_became_active":"2024-04-28T12:37:26.867663+0000","last_became_peered":"2024-04-28T12:37:26.867663+0000","last_unstale":"2024-04-28T12:37:27.846043+0000","last_undegraded":"2024-04-28T12:37:27.846043+0000","last_fullsized":"2024-04-28T12:37:27.846043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:45.428951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978823+0000","last_change":"2024-04-28T12:37:22.797968+0000","last_active":"2024-04-28T12:37:27.978823+0000","last_peered":"2024-04-28T12:37:27.978823+0000","last_clean":"2024-04-28T12:37:27.978823+0000","last_became_active":"2024-04-28T12:37:22.797258+0000","last_became_peered":"2024-04-28T12:37:22.797258+0000","last_unstale":"2024-04-28T12:37:27.978823+0000","last_undegraded":"2024-04-28T12:37:27.978823+0000","last_fullsized":"2024-04-28T12:37:27.978823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843883+0000","last_change":"2024-04-28T12:37:21.814135+0000","last_active":"2024-04-28T12:37:27.843883+0000","last_peered":"2024-04-28T12:37:27.843883+0000","last_clean":"2024-04-28T12:37:27.843883+0000","last_became_active":"2024-04-28T12:37:21.813800+0000","last_became_peered":"2024-04-28T12:37:21.813800+0000","last_unstale":"2024-04-28T12:37:27.843883+0000","last_undegraded":"2024-04-28T12:37:27.843883+0000","last_fullsized":"2024-04-28T12:37:27.843883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:49.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844709+0000","last_change":"2024-04-28T12:37:26.885109+0000","last_active":"2024-04-28T12:37:27.844709+0000","last_peered":"2024-04-28T12:37:27.844709+0000","last_clean":"2024-04-28T12:37:27.844709+0000","last_became_active":"2024-04-28T12:37:26.882922+0000","last_became_peered":"2024-04-28T12:37:26.882922+0000","last_unstale":"2024-04-28T12:37:27.844709+0000","last_undegraded":"2024-04-28T12:37:27.844709+0000","last_fullsized":"2024-04-28T12:37:27.844709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:20.147514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978601+0000","last_change":"2024-04-28T12:37:22.802331+0000","last_active":"2024-04-28T12:37:27.978601+0000","last_peered":"2024-04-28T12:37:27.978601+0000","last_clean":"2024-04-28T12:37:27.978601+0000","last_became_active":"2024-04-28T12:37:22.801939+0000","last_became_peered":"2024-04-28T12:37:22.801939+0000","last_unstale":"2024-04-28T12:37:27.978601+0000","last_undegraded":"2024-04-28T12:37:27.978601+0000","last_fullsized":"2024-04-28T12:37:27.978601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:59.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976553+0000","last_change":"2024-04-28T12:37:21.796110+0000","last_active":"2024-04-28T12:37:27.976553+0000","last_peered":"2024-04-28T12:37:27.976553+0000","last_clean":"2024-04-28T12:37:27.976553+0000","last_became_active":"2024-04-28T12:37:21.795405+0000","last_became_peered":"2024-04-28T12:37:21.795405+0000","last_unstale":"2024-04-28T12:37:27.976553+0000","last_undegraded":"2024-04-28T12:37:27.976553+0000","last_fullsized":"2024-04-28T12:37:27.976553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:53.650061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976613+0000","last_change":"2024-04-28T12:37:26.883160+0000","last_active":"2024-04-28T12:37:27.976613+0000","last_peered":"2024-04-28T12:37:27.976613+0000","last_clean":"2024-04-28T12:37:27.976613+0000","last_became_active":"2024-04-28T12:37:26.882684+0000","last_became_peered":"2024-04-28T12:37:26.882684+0000","last_unstale":"2024-04-28T12:37:27.976613+0000","last_undegraded":"2024-04-28T12:37:27.976613+0000","last_fullsized":"2024-04-28T12:37:27.976613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:19:01.590267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981129+0000","last_change":"2024-04-28T12:37:22.800581+0000","last_active":"2024-04-28T12:37:27.981129+0000","last_peered":"2024-04-28T12:37:27.981129+0000","last_clean":"2024-04-28T12:37:27.981129+0000","last_became_active":"2024-04-28T12:37:22.800241+0000","last_became_peered":"2024-04-28T12:37:22.800241+0000","last_unstale":"2024-04-28T12:37:27.981129+0000","last_undegraded":"2024-04-28T12:37:27.981129+0000","last_fullsized":"2024-04-28T12:37:27.981129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982900+0000","last_change":"2024-04-28T12:37:21.811668+0000","last_active":"2024-04-28T12:37:27.982900+0000","last_peered":"2024-04-28T12:37:27.982900+0000","last_clean":"2024-04-28T12:37:27.982900+0000","last_became_active":"2024-04-28T12:37:21.811372+0000","last_became_peered":"2024-04-28T12:37:21.811372+0000","last_unstale":"2024-04-28T12:37:27.982900+0000","last_undegraded":"2024-04-28T12:37:27.982900+0000","last_fullsized":"2024-04-28T12:37:27.982900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:00:04.591555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976863+0000","last_change":"2024-04-28T12:37:26.877753+0000","last_active":"2024-04-28T12:37:27.976863+0000","last_peered":"2024-04-28T12:37:27.976863+0000","last_clean":"2024-04-28T12:37:27.976863+0000","last_became_active":"2024-04-28T12:37:26.877456+0000","last_became_peered":"2024-04-28T12:37:26.877456+0000","last_unstale":"2024-04-28T12:37:27.976863+0000","last_undegraded":"2024-04-28T12:37:27.976863+0000","last_fullsized":"2024-04-28T12:37:27.976863+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:45.428951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979882+0000","last_change":"2024-04-28T12:37:22.801683+0000","last_active":"2024-04-28T12:37:27.979882+0000","last_peered":"2024-04-28T12:37:27.979882+0000","last_clean":"2024-04-28T12:37:27.979882+0000","last_became_active":"2024-04-28T12:37:22.801378+0000","last_became_peered":"2024-04-28T12:37:22.801378+0000","last_unstale":"2024-04-28T12:37:27.979882+0000","last_undegraded":"2024-04-28T12:37:27.979882+0000","last_fullsized":"2024-04-28T12:37:27.979882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:41.371396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981252+0000","last_change":"2024-04-28T12:37:21.791074+0000","last_active":"2024-04-28T12:37:27.981252+0000","last_peered":"2024-04-28T12:37:27.981252+0000","last_clean":"2024-04-28T12:37:27.981252+0000","last_became_active":"2024-04-28T12:37:21.790645+0000","last_became_peered":"2024-04-28T12:37:21.790645+0000","last_unstale":"2024-04-28T12:37:27.981252+0000","last_undegraded":"2024-04-28T12:37:27.981252+0000","last_fullsized":"2024-04-28T12:37:27.981252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977495+0000","last_change":"2024-04-28T12:37:26.873652+0000","last_active":"2024-04-28T12:37:27.977495+0000","last_peered":"2024-04-28T12:37:27.977495+0000","last_clean":"2024-04-28T12:37:27.977495+0000","last_became_active":"2024-04-28T12:37:26.873065+0000","last_became_peered":"2024-04-28T12:37:26.873065+0000","last_unstale":"2024-04-28T12:37:27.977495+0000","last_undegraded":"2024-04-28T12:37:27.977495+0000","last_fullsized":"2024-04-28T12:37:27.977495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845437+0000","last_change":"2024-04-28T12:37:22.794106+0000","last_active":"2024-04-28T12:37:27.845437+0000","last_peered":"2024-04-28T12:37:27.845437+0000","last_clean":"2024-04-28T12:37:27.845437+0000","last_became_active":"2024-04-28T12:37:22.789651+0000","last_became_peered":"2024-04-28T12:37:22.789651+0000","last_unstale":"2024-04-28T12:37:27.845437+0000","last_undegraded":"2024-04-28T12:37:27.845437+0000","last_fullsized":"2024-04-28T12:37:27.845437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:33.269962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979186+0000","last_change":"2024-04-28T12:37:21.810116+0000","last_active":"2024-04-28T12:37:27.979186+0000","last_peered":"2024-04-28T12:37:27.979186+0000","last_clean":"2024-04-28T12:37:27.979186+0000","last_became_active":"2024-04-28T12:37:21.809736+0000","last_became_peered":"2024-04-28T12:37:21.809736+0000","last_unstale":"2024-04-28T12:37:27.979186+0000","last_undegraded":"2024-04-28T12:37:27.979186+0000","last_fullsized":"2024-04-28T12:37:27.979186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:39.334819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981752+0000","last_change":"2024-04-28T12:37:26.867239+0000","last_active":"2024-04-28T12:37:27.981752+0000","last_peered":"2024-04-28T12:37:27.981752+0000","last_clean":"2024-04-28T12:37:27.981752+0000","last_became_active":"2024-04-28T12:37:26.866848+0000","last_became_peered":"2024-04-28T12:37:26.866848+0000","last_unstale":"2024-04-28T12:37:27.981752+0000","last_undegraded":"2024-04-28T12:37:27.981752+0000","last_fullsized":"2024-04-28T12:37:27.981752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:08:51.032543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976301+0000","last_change":"2024-04-28T12:37:22.796512+0000","last_active":"2024-04-28T12:37:27.976301+0000","last_peered":"2024-04-28T12:37:27.976301+0000","last_clean":"2024-04-28T12:37:27.976301+0000","last_became_active":"2024-04-28T12:37:22.793368+0000","last_became_peered":"2024-04-28T12:37:22.793368+0000","last_unstale":"2024-04-28T12:37:27.976301+0000","last_undegraded":"2024-04-28T12:37:27.976301+0000","last_fullsized":"2024-04-28T12:37:27.976301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.839137+0000","last_change":"2024-04-28T12:37:21.793644+0000","last_active":"2024-04-28T12:37:27.839137+0000","last_peered":"2024-04-28T12:37:27.839137+0000","last_clean":"2024-04-28T12:37:27.839137+0000","last_became_active":"2024-04-28T12:37:21.793358+0000","last_became_peered":"2024-04-28T12:37:21.793358+0000","last_unstale":"2024-04-28T12:37:27.839137+0000","last_undegraded":"2024-04-28T12:37:27.839137+0000","last_fullsized":"2024-04-28T12:37:27.839137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:55:28.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978474+0000","last_change":"2024-04-28T12:37:26.866571+0000","last_active":"2024-04-28T12:37:27.978474+0000","last_peered":"2024-04-28T12:37:27.978474+0000","last_clean":"2024-04-28T12:37:27.978474+0000","last_became_active":"2024-04-28T12:37:26.866042+0000","last_became_peered":"2024-04-28T12:37:26.866042+0000","last_unstale":"2024-04-28T12:37:27.978474+0000","last_undegraded":"2024-04-28T12:37:27.978474+0000","last_fullsized":"2024-04-28T12:37:27.978474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:04.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976223+0000","last_change":"2024-04-28T12:37:22.796747+0000","last_active":"2024-04-28T12:37:27.976223+0000","last_peered":"2024-04-28T12:37:27.976223+0000","last_clean":"2024-04-28T12:37:27.976223+0000","last_became_active":"2024-04-28T12:37:22.795677+0000","last_became_peered":"2024-04-28T12:37:22.795677+0000","last_unstale":"2024-04-28T12:37:27.976223+0000","last_undegraded":"2024-04-28T12:37:27.976223+0000","last_fullsized":"2024-04-28T12:37:27.976223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:05:18.644675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979595+0000","last_change":"2024-04-28T12:37:21.814725+0000","last_active":"2024-04-28T12:37:27.979595+0000","last_peered":"2024-04-28T12:37:27.979595+0000","last_clean":"2024-04-28T12:37:27.979595+0000","last_became_active":"2024-04-28T12:37:21.814516+0000","last_became_peered":"2024-04-28T12:37:21.814516+0000","last_unstale":"2024-04-28T12:37:27.979595+0000","last_undegraded":"2024-04-28T12:37:27.979595+0000","last_fullsized":"2024-04-28T12:37:27.979595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:49.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978718+0000","last_change":"2024-04-28T12:37:26.886450+0000","last_active":"2024-04-28T12:37:27.978718+0000","last_peered":"2024-04-28T12:37:27.978718+0000","last_clean":"2024-04-28T12:37:27.978718+0000","last_became_active":"2024-04-28T12:37:26.883398+0000","last_became_peered":"2024-04-28T12:37:26.883398+0000","last_unstale":"2024-04-28T12:37:27.978718+0000","last_undegraded":"2024-04-28T12:37:27.978718+0000","last_fullsized":"2024-04-28T12:37:27.978718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:04.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977534+0000","last_change":"2024-04-28T12:37:22.814159+0000","last_active":"2024-04-28T12:37:27.977534+0000","last_peered":"2024-04-28T12:37:27.977534+0000","last_clean":"2024-04-28T12:37:27.977534+0000","last_became_active":"2024-04-28T12:37:22.813889+0000","last_became_peered":"2024-04-28T12:37:22.813889+0000","last_unstale":"2024-04-28T12:37:27.977534+0000","last_undegraded":"2024-04-28T12:37:27.977534+0000","last_fullsized":"2024-04-28T12:37:27.977534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:16.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979063+0000","last_change":"2024-04-28T12:37:21.803169+0000","last_active":"2024-04-28T12:37:27.979063+0000","last_peered":"2024-04-28T12:37:27.979063+0000","last_clean":"2024-04-28T12:37:27.979063+0000","last_became_active":"2024-04-28T12:37:21.800812+0000","last_became_peered":"2024-04-28T12:37:21.800812+0000","last_unstale":"2024-04-28T12:37:27.979063+0000","last_undegraded":"2024-04-28T12:37:27.979063+0000","last_fullsized":"2024-04-28T12:37:27.979063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:55:28.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979702+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:37:27.979702+0000","last_peered":"2024-04-28T12:37:27.979702+0000","last_clean":"2024-04-28T12:37:27.979702+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:37:27.979702+0000","last_undegraded":"2024-04-28T12:37:27.979702+0000","last_fullsized":"2024-04-28T12:37:27.979702+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843882+0000","last_change":"2024-04-28T12:37:26.884731+0000","last_active":"2024-04-28T12:37:27.843882+0000","last_peered":"2024-04-28T12:37:27.843882+0000","last_clean":"2024-04-28T12:37:27.843882+0000","last_became_active":"2024-04-28T12:37:26.882799+0000","last_became_peered":"2024-04-28T12:37:26.882799+0000","last_unstale":"2024-04-28T12:37:27.843882+0000","last_undegraded":"2024-04-28T12:37:27.843882+0000","last_fullsized":"2024-04-28T12:37:27.843882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:51.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982421+0000","last_change":"2024-04-28T12:37:22.818975+0000","last_active":"2024-04-28T12:37:27.982421+0000","last_peered":"2024-04-28T12:37:27.982421+0000","last_clean":"2024-04-28T12:37:27.982421+0000","last_became_active":"2024-04-28T12:37:22.818677+0000","last_became_peered":"2024-04-28T12:37:22.818677+0000","last_unstale":"2024-04-28T12:37:27.982421+0000","last_undegraded":"2024-04-28T12:37:27.982421+0000","last_fullsized":"2024-04-28T12:37:27.982421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:41.371396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840221+0000","last_change":"2024-04-28T12:37:21.800771+0000","last_active":"2024-04-28T12:37:27.840221+0000","last_peered":"2024-04-28T12:37:27.840221+0000","last_clean":"2024-04-28T12:37:27.840221+0000","last_became_active":"2024-04-28T12:37:21.800451+0000","last_became_peered":"2024-04-28T12:37:21.800451+0000","last_unstale":"2024-04-28T12:37:27.840221+0000","last_undegraded":"2024-04-28T12:37:27.840221+0000","last_fullsized":"2024-04-28T12:37:27.840221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842666+0000","last_change":"2024-04-28T12:37:26.879284+0000","last_active":"2024-04-28T12:37:27.842666+0000","last_peered":"2024-04-28T12:37:27.842666+0000","last_clean":"2024-04-28T12:37:27.842666+0000","last_became_active":"2024-04-28T12:37:26.878864+0000","last_became_peered":"2024-04-28T12:37:26.878864+0000","last_unstale":"2024-04-28T12:37:27.842666+0000","last_undegraded":"2024-04-28T12:37:27.842666+0000","last_fullsized":"2024-04-28T12:37:27.842666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:45.428951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979686+0000","last_change":"2024-04-28T12:37:22.824563+0000","last_active":"2024-04-28T12:37:27.979686+0000","last_peered":"2024-04-28T12:37:27.979686+0000","last_clean":"2024-04-28T12:37:27.979686+0000","last_became_active":"2024-04-28T12:37:22.818493+0000","last_became_peered":"2024-04-28T12:37:22.818493+0000","last_unstale":"2024-04-28T12:37:27.979686+0000","last_undegraded":"2024-04-28T12:37:27.979686+0000","last_fullsized":"2024-04-28T12:37:27.979686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:00.502362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976419+0000","last_change":"2024-04-28T12:37:21.799906+0000","last_active":"2024-04-28T12:37:27.976419+0000","last_peered":"2024-04-28T12:37:27.976419+0000","last_clean":"2024-04-28T12:37:27.976419+0000","last_became_active":"2024-04-28T12:37:21.799470+0000","last_became_peered":"2024-04-28T12:37:21.799470+0000","last_unstale":"2024-04-28T12:37:27.976419+0000","last_undegraded":"2024-04-28T12:37:27.976419+0000","last_fullsized":"2024-04-28T12:37:27.976419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:49.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977214+0000","last_change":"2024-04-28T12:37:26.874939+0000","last_active":"2024-04-28T12:37:27.977214+0000","last_peered":"2024-04-28T12:37:27.977214+0000","last_clean":"2024-04-28T12:37:27.977214+0000","last_became_active":"2024-04-28T12:37:26.874472+0000","last_became_peered":"2024-04-28T12:37:26.874472+0000","last_unstale":"2024-04-28T12:37:27.977214+0000","last_undegraded":"2024-04-28T12:37:27.977214+0000","last_fullsized":"2024-04-28T12:37:27.977214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981950+0000","last_change":"2024-04-28T12:37:22.814584+0000","last_active":"2024-04-28T12:37:27.981950+0000","last_peered":"2024-04-28T12:37:27.981950+0000","last_clean":"2024-04-28T12:37:27.981950+0000","last_became_active":"2024-04-28T12:37:22.812752+0000","last_became_peered":"2024-04-28T12:37:22.812752+0000","last_unstale":"2024-04-28T12:37:27.981950+0000","last_undegraded":"2024-04-28T12:37:27.981950+0000","last_fullsized":"2024-04-28T12:37:27.981950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:48:32.880830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840416+0000","last_change":"2024-04-28T12:37:21.801322+0000","last_active":"2024-04-28T12:37:27.840416+0000","last_peered":"2024-04-28T12:37:27.840416+0000","last_clean":"2024-04-28T12:37:27.840416+0000","last_became_active":"2024-04-28T12:37:21.800437+0000","last_became_peered":"2024-04-28T12:37:21.800437+0000","last_unstale":"2024-04-28T12:37:27.840416+0000","last_undegraded":"2024-04-28T12:37:27.840416+0000","last_fullsized":"2024-04-28T12:37:27.840416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:33.616030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978163+0000","last_change":"2024-04-28T12:37:26.885854+0000","last_active":"2024-04-28T12:37:27.978163+0000","last_peered":"2024-04-28T12:37:27.978163+0000","last_clean":"2024-04-28T12:37:27.978163+0000","last_became_active":"2024-04-28T12:37:26.884298+0000","last_became_peered":"2024-04-28T12:37:26.884298+0000","last_unstale":"2024-04-28T12:37:27.978163+0000","last_undegraded":"2024-04-28T12:37:27.978163+0000","last_fullsized":"2024-04-28T12:37:27.978163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:19:01.590267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982107+0000","last_change":"2024-04-28T12:37:22.808698+0000","last_active":"2024-04-28T12:37:27.982107+0000","last_peered":"2024-04-28T12:37:27.982107+0000","last_clean":"2024-04-28T12:37:27.982107+0000","last_became_active":"2024-04-28T12:37:22.808363+0000","last_became_peered":"2024-04-28T12:37:22.808363+0000","last_unstale":"2024-04-28T12:37:27.982107+0000","last_undegraded":"2024-04-28T12:37:27.982107+0000","last_fullsized":"2024-04-28T12:37:27.982107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:47.618755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.846296+0000","last_change":"2024-04-28T12:37:21.794490+0000","last_active":"2024-04-28T12:37:27.846296+0000","last_peered":"2024-04-28T12:37:27.846296+0000","last_clean":"2024-04-28T12:37:27.846296+0000","last_became_active":"2024-04-28T12:37:21.794115+0000","last_became_peered":"2024-04-28T12:37:21.794115+0000","last_unstale":"2024-04-28T12:37:27.846296+0000","last_undegraded":"2024-04-28T12:37:27.846296+0000","last_fullsized":"2024-04-28T12:37:27.846296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:20:04.188478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844144+0000","last_change":"2024-04-28T12:37:26.867905+0000","last_active":"2024-04-28T12:37:27.844144+0000","last_peered":"2024-04-28T12:37:27.844144+0000","last_clean":"2024-04-28T12:37:27.844144+0000","last_became_active":"2024-04-28T12:37:26.867317+0000","last_became_peered":"2024-04-28T12:37:26.867317+0000","last_unstale":"2024-04-28T12:37:27.844144+0000","last_undegraded":"2024-04-28T12:37:27.844144+0000","last_fullsized":"2024-04-28T12:37:27.844144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:49.659072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980575+0000","last_change":"2024-04-28T12:37:22.802406+0000","last_active":"2024-04-28T12:37:27.980575+0000","last_peered":"2024-04-28T12:37:27.980575+0000","last_clean":"2024-04-28T12:37:27.980575+0000","last_became_active":"2024-04-28T12:37:22.802094+0000","last_became_peered":"2024-04-28T12:37:22.802094+0000","last_unstale":"2024-04-28T12:37:27.980575+0000","last_undegraded":"2024-04-28T12:37:27.980575+0000","last_fullsized":"2024-04-28T12:37:27.980575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:16.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.838661+0000","last_change":"2024-04-28T12:37:21.788140+0000","last_active":"2024-04-28T12:37:27.838661+0000","last_peered":"2024-04-28T12:37:27.838661+0000","last_clean":"2024-04-28T12:37:27.838661+0000","last_became_active":"2024-04-28T12:37:21.787727+0000","last_became_peered":"2024-04-28T12:37:21.787727+0000","last_unstale":"2024-04-28T12:37:27.838661+0000","last_undegraded":"2024-04-28T12:37:27.838661+0000","last_fullsized":"2024-04-28T12:37:27.838661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:29:20.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978548+0000","last_change":"2024-04-28T12:37:26.868358+0000","last_active":"2024-04-28T12:37:27.978548+0000","last_peered":"2024-04-28T12:37:27.978548+0000","last_clean":"2024-04-28T12:37:27.978548+0000","last_became_active":"2024-04-28T12:37:26.867972+0000","last_became_peered":"2024-04-28T12:37:26.867972+0000","last_unstale":"2024-04-28T12:37:27.978548+0000","last_undegraded":"2024-04-28T12:37:27.978548+0000","last_fullsized":"2024-04-28T12:37:27.978548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:28.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842120+0000","last_change":"2024-04-28T12:37:22.786192+0000","last_active":"2024-04-28T12:37:27.842120+0000","last_peered":"2024-04-28T12:37:27.842120+0000","last_clean":"2024-04-28T12:37:27.842120+0000","last_became_active":"2024-04-28T12:37:22.785730+0000","last_became_peered":"2024-04-28T12:37:22.785730+0000","last_unstale":"2024-04-28T12:37:27.842120+0000","last_undegraded":"2024-04-28T12:37:27.842120+0000","last_fullsized":"2024-04-28T12:37:27.842120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:42.893320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842909+0000","last_change":"2024-04-28T12:37:21.800159+0000","last_active":"2024-04-28T12:37:27.842909+0000","last_peered":"2024-04-28T12:37:27.842909+0000","last_clean":"2024-04-28T12:37:27.842909+0000","last_became_active":"2024-04-28T12:37:21.799817+0000","last_became_peered":"2024-04-28T12:37:21.799817+0000","last_unstale":"2024-04-28T12:37:27.842909+0000","last_undegraded":"2024-04-28T12:37:27.842909+0000","last_fullsized":"2024-04-28T12:37:27.842909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978507+0000","last_change":"2024-04-28T12:37:26.883264+0000","last_active":"2024-04-28T12:37:27.978507+0000","last_peered":"2024-04-28T12:37:27.978507+0000","last_clean":"2024-04-28T12:37:27.978507+0000","last_became_active":"2024-04-28T12:37:26.882787+0000","last_became_peered":"2024-04-28T12:37:26.882787+0000","last_unstale":"2024-04-28T12:37:27.978507+0000","last_undegraded":"2024-04-28T12:37:27.978507+0000","last_fullsized":"2024-04-28T12:37:27.978507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:11:24.219747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981395+0000","last_change":"2024-04-28T12:37:22.811865+0000","last_active":"2024-04-28T12:37:27.981395+0000","last_peered":"2024-04-28T12:37:27.981395+0000","last_clean":"2024-04-28T12:37:27.981395+0000","last_became_active":"2024-04-28T12:37:22.809479+0000","last_became_peered":"2024-04-28T12:37:22.809479+0000","last_unstale":"2024-04-28T12:37:27.981395+0000","last_undegraded":"2024-04-28T12:37:27.981395+0000","last_fullsized":"2024-04-28T12:37:27.981395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:39.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978873+0000","last_change":"2024-04-28T12:37:21.805620+0000","last_active":"2024-04-28T12:37:27.978873+0000","last_peered":"2024-04-28T12:37:27.978873+0000","last_clean":"2024-04-28T12:37:27.978873+0000","last_became_active":"2024-04-28T12:37:21.805397+0000","last_became_peered":"2024-04-28T12:37:21.805397+0000","last_unstale":"2024-04-28T12:37:27.978873+0000","last_undegraded":"2024-04-28T12:37:27.978873+0000","last_fullsized":"2024-04-28T12:37:27.978873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978474+0000","last_change":"2024-04-28T12:37:26.880544+0000","last_active":"2024-04-28T12:37:27.978474+0000","last_peered":"2024-04-28T12:37:27.978474+0000","last_clean":"2024-04-28T12:37:27.978474+0000","last_became_active":"2024-04-28T12:37:26.878960+0000","last_became_peered":"2024-04-28T12:37:26.878960+0000","last_unstale":"2024-04-28T12:37:27.978474+0000","last_undegraded":"2024-04-28T12:37:27.978474+0000","last_fullsized":"2024-04-28T12:37:27.978474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:28:21.376871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977145+0000","last_change":"2024-04-28T12:37:22.795607+0000","last_active":"2024-04-28T12:37:27.977145+0000","last_peered":"2024-04-28T12:37:27.977145+0000","last_clean":"2024-04-28T12:37:27.977145+0000","last_became_active":"2024-04-28T12:37:22.795262+0000","last_became_peered":"2024-04-28T12:37:22.795262+0000","last_unstale":"2024-04-28T12:37:27.977145+0000","last_undegraded":"2024-04-28T12:37:27.977145+0000","last_fullsized":"2024-04-28T12:37:27.977145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:42.893320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977438+0000","last_change":"2024-04-28T12:37:21.798231+0000","last_active":"2024-04-28T12:37:27.977438+0000","last_peered":"2024-04-28T12:37:27.977438+0000","last_clean":"2024-04-28T12:37:27.977438+0000","last_became_active":"2024-04-28T12:37:21.797978+0000","last_became_peered":"2024-04-28T12:37:21.797978+0000","last_unstale":"2024-04-28T12:37:27.977438+0000","last_undegraded":"2024-04-28T12:37:27.977438+0000","last_fullsized":"2024-04-28T12:37:27.977438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:20:04.188478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980535+0000","last_change":"2024-04-28T12:37:26.881273+0000","last_active":"2024-04-28T12:37:27.980535+0000","last_peered":"2024-04-28T12:37:27.980535+0000","last_clean":"2024-04-28T12:37:27.980535+0000","last_became_active":"2024-04-28T12:37:26.880845+0000","last_became_peered":"2024-04-28T12:37:26.880845+0000","last_unstale":"2024-04-28T12:37:27.980535+0000","last_undegraded":"2024-04-28T12:37:27.980535+0000","last_fullsized":"2024-04-28T12:37:27.980535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:09.634939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981232+0000","last_change":"2024-04-28T12:37:22.811901+0000","last_active":"2024-04-28T12:37:27.981232+0000","last_peered":"2024-04-28T12:37:27.981232+0000","last_clean":"2024-04-28T12:37:27.981232+0000","last_became_active":"2024-04-28T12:37:22.807998+0000","last_became_peered":"2024-04-28T12:37:22.807998+0000","last_unstale":"2024-04-28T12:37:27.981232+0000","last_undegraded":"2024-04-28T12:37:27.981232+0000","last_fullsized":"2024-04-28T12:37:27.981232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:29:25.559764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:41.723137+0000","last_change":"2024-04-28T12:37:21.802895+0000","last_active":"2024-04-28T12:37:41.723137+0000","last_peered":"2024-04-28T12:37:41.723137+0000","last_clean":"2024-04-28T12:37:41.723137+0000","last_became_active":"2024-04-28T12:37:21.801912+0000","last_became_peered":"2024-04-28T12:37:21.801912+0000","last_unstale":"2024-04-28T12:37:41.723137+0000","last_undegraded":"2024-04-28T12:37:41.723137+0000","last_fullsized":"2024-04-28T12:37:41.723137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:39.334819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977722+0000","last_change":"2024-04-28T12:37:26.873444+0000","last_active":"2024-04-28T12:37:27.977722+0000","last_peered":"2024-04-28T12:37:27.977722+0000","last_clean":"2024-04-28T12:37:27.977722+0000","last_became_active":"2024-04-28T12:37:26.872331+0000","last_became_peered":"2024-04-28T12:37:26.872331+0000","last_unstale":"2024-04-28T12:37:27.977722+0000","last_undegraded":"2024-04-28T12:37:27.977722+0000","last_fullsized":"2024-04-28T12:37:27.977722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:51.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845884+0000","last_change":"2024-04-28T12:37:22.786473+0000","last_active":"2024-04-28T12:37:27.845884+0000","last_peered":"2024-04-28T12:37:27.845884+0000","last_clean":"2024-04-28T12:37:27.845884+0000","last_became_active":"2024-04-28T12:37:22.785266+0000","last_became_peered":"2024-04-28T12:37:22.785266+0000","last_unstale":"2024-04-28T12:37:27.845884+0000","last_undegraded":"2024-04-28T12:37:27.845884+0000","last_fullsized":"2024-04-28T12:37:27.845884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:50.704472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976275+0000","last_change":"2024-04-28T12:37:21.810514+0000","last_active":"2024-04-28T12:37:27.976275+0000","last_peered":"2024-04-28T12:37:27.976275+0000","last_clean":"2024-04-28T12:37:27.976275+0000","last_became_active":"2024-04-28T12:37:21.810221+0000","last_became_peered":"2024-04-28T12:37:21.810221+0000","last_unstale":"2024-04-28T12:37:27.976275+0000","last_undegraded":"2024-04-28T12:37:27.976275+0000","last_fullsized":"2024-04-28T12:37:27.976275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845261+0000","last_change":"2024-04-28T12:37:26.871384+0000","last_active":"2024-04-28T12:37:27.845261+0000","last_peered":"2024-04-28T12:37:27.845261+0000","last_clean":"2024-04-28T12:37:27.845261+0000","last_became_active":"2024-04-28T12:37:26.868701+0000","last_became_peered":"2024-04-28T12:37:26.868701+0000","last_unstale":"2024-04-28T12:37:27.845261+0000","last_undegraded":"2024-04-28T12:37:27.845261+0000","last_fullsized":"2024-04-28T12:37:27.845261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:44.015884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979855+0000","last_change":"2024-04-28T12:37:22.821706+0000","last_active":"2024-04-28T12:37:27.979855+0000","last_peered":"2024-04-28T12:37:27.979855+0000","last_clean":"2024-04-28T12:37:27.979855+0000","last_became_active":"2024-04-28T12:37:22.821338+0000","last_became_peered":"2024-04-28T12:37:22.821338+0000","last_unstale":"2024-04-28T12:37:27.979855+0000","last_undegraded":"2024-04-28T12:37:27.979855+0000","last_fullsized":"2024-04-28T12:37:27.979855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980054+0000","last_change":"2024-04-28T12:37:21.803535+0000","last_active":"2024-04-28T12:37:27.980054+0000","last_peered":"2024-04-28T12:37:27.980054+0000","last_clean":"2024-04-28T12:37:27.980054+0000","last_became_active":"2024-04-28T12:37:21.801895+0000","last_became_peered":"2024-04-28T12:37:21.801895+0000","last_unstale":"2024-04-28T12:37:27.980054+0000","last_undegraded":"2024-04-28T12:37:27.980054+0000","last_fullsized":"2024-04-28T12:37:27.980054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:32.253270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976957+0000","last_change":"2024-04-28T12:37:26.861154+0000","last_active":"2024-04-28T12:37:27.976957+0000","last_peered":"2024-04-28T12:37:27.976957+0000","last_clean":"2024-04-28T12:37:27.976957+0000","last_became_active":"2024-04-28T12:37:26.860732+0000","last_became_peered":"2024-04-28T12:37:26.860732+0000","last_unstale":"2024-04-28T12:37:27.976957+0000","last_undegraded":"2024-04-28T12:37:27.976957+0000","last_fullsized":"2024-04-28T12:37:27.976957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:48:40.360635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979341+0000","last_change":"2024-04-28T12:37:22.817129+0000","last_active":"2024-04-28T12:37:27.979341+0000","last_peered":"2024-04-28T12:37:27.979341+0000","last_clean":"2024-04-28T12:37:27.979341+0000","last_became_active":"2024-04-28T12:37:22.816811+0000","last_became_peered":"2024-04-28T12:37:22.816811+0000","last_unstale":"2024-04-28T12:37:27.979341+0000","last_undegraded":"2024-04-28T12:37:27.979341+0000","last_fullsized":"2024-04-28T12:37:27.979341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:29:25.559764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843391+0000","last_change":"2024-04-28T12:37:21.805873+0000","last_active":"2024-04-28T12:37:27.843391+0000","last_peered":"2024-04-28T12:37:27.843391+0000","last_clean":"2024-04-28T12:37:27.843391+0000","last_became_active":"2024-04-28T12:37:21.805549+0000","last_became_peered":"2024-04-28T12:37:21.805549+0000","last_unstale":"2024-04-28T12:37:27.843391+0000","last_undegraded":"2024-04-28T12:37:27.843391+0000","last_fullsized":"2024-04-28T12:37:27.843391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976927+0000","last_change":"2024-04-28T12:37:26.883208+0000","last_active":"2024-04-28T12:37:27.976927+0000","last_peered":"2024-04-28T12:37:27.976927+0000","last_clean":"2024-04-28T12:37:27.976927+0000","last_became_active":"2024-04-28T12:37:26.882732+0000","last_became_peered":"2024-04-28T12:37:26.882732+0000","last_unstale":"2024-04-28T12:37:27.976927+0000","last_undegraded":"2024-04-28T12:37:27.976927+0000","last_fullsized":"2024-04-28T12:37:27.976927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:37:09.856024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978306+0000","last_change":"2024-04-28T12:37:26.888160+0000","last_active":"2024-04-28T12:37:27.978306+0000","last_peered":"2024-04-28T12:37:27.978306+0000","last_clean":"2024-04-28T12:37:27.978306+0000","last_became_active":"2024-04-28T12:37:26.886631+0000","last_became_peered":"2024-04-28T12:37:26.886631+0000","last_unstale":"2024-04-28T12:37:27.978306+0000","last_undegraded":"2024-04-28T12:37:27.978306+0000","last_fullsized":"2024-04-28T12:37:27.978306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:51.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.975748+0000","last_change":"2024-04-28T12:37:22.794465+0000","last_active":"2024-04-28T12:37:27.975748+0000","last_peered":"2024-04-28T12:37:27.975748+0000","last_clean":"2024-04-28T12:37:27.975748+0000","last_became_active":"2024-04-28T12:37:22.794138+0000","last_became_peered":"2024-04-28T12:37:22.794138+0000","last_unstale":"2024-04-28T12:37:27.975748+0000","last_undegraded":"2024-04-28T12:37:27.975748+0000","last_fullsized":"2024-04-28T12:37:27.975748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978354+0000","last_change":"2024-04-28T12:37:21.805697+0000","last_active":"2024-04-28T12:37:27.978354+0000","last_peered":"2024-04-28T12:37:27.978354+0000","last_clean":"2024-04-28T12:37:27.978354+0000","last_became_active":"2024-04-28T12:37:21.805449+0000","last_became_peered":"2024-04-28T12:37:21.805449+0000","last_unstale":"2024-04-28T12:37:27.978354+0000","last_undegraded":"2024-04-28T12:37:27.978354+0000","last_fullsized":"2024-04-28T12:37:27.978354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843785+0000","last_change":"2024-04-28T12:37:26.869931+0000","last_active":"2024-04-28T12:37:27.843785+0000","last_peered":"2024-04-28T12:37:27.843785+0000","last_clean":"2024-04-28T12:37:27.843785+0000","last_became_active":"2024-04-28T12:37:26.869441+0000","last_became_peered":"2024-04-28T12:37:26.869441+0000","last_unstale":"2024-04-28T12:37:27.843785+0000","last_undegraded":"2024-04-28T12:37:27.843785+0000","last_fullsized":"2024-04-28T12:37:27.843785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977035+0000","last_change":"2024-04-28T12:37:22.792490+0000","last_active":"2024-04-28T12:37:27.977035+0000","last_peered":"2024-04-28T12:37:27.977035+0000","last_clean":"2024-04-28T12:37:27.977035+0000","last_became_active":"2024-04-28T12:37:22.790799+0000","last_became_peered":"2024-04-28T12:37:22.790799+0000","last_unstale":"2024-04-28T12:37:27.977035+0000","last_undegraded":"2024-04-28T12:37:27.977035+0000","last_fullsized":"2024-04-28T12:37:27.977035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977471+0000","last_change":"2024-04-28T12:37:21.799595+0000","last_active":"2024-04-28T12:37:27.977471+0000","last_peered":"2024-04-28T12:37:27.977471+0000","last_clean":"2024-04-28T12:37:27.977471+0000","last_became_active":"2024-04-28T12:37:21.796500+0000","last_became_peered":"2024-04-28T12:37:21.796500+0000","last_unstale":"2024-04-28T12:37:27.977471+0000","last_undegraded":"2024-04-28T12:37:27.977471+0000","last_fullsized":"2024-04-28T12:37:27.977471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:33.616030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843252+0000","last_change":"2024-04-28T12:37:21.799592+0000","last_active":"2024-04-28T12:37:27.843252+0000","last_peered":"2024-04-28T12:37:27.843252+0000","last_clean":"2024-04-28T12:37:27.843252+0000","last_became_active":"2024-04-28T12:37:21.799085+0000","last_became_peered":"2024-04-28T12:37:21.799085+0000","last_unstale":"2024-04-28T12:37:27.843252+0000","last_undegraded":"2024-04-28T12:37:27.843252+0000","last_fullsized":"2024-04-28T12:37:27.843252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:53.650061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977307+0000","last_change":"2024-04-28T12:37:22.781856+0000","last_active":"2024-04-28T12:37:27.977307+0000","last_peered":"2024-04-28T12:37:27.977307+0000","last_clean":"2024-04-28T12:37:27.977307+0000","last_became_active":"2024-04-28T12:37:22.780347+0000","last_became_peered":"2024-04-28T12:37:22.780347+0000","last_unstale":"2024-04-28T12:37:27.977307+0000","last_undegraded":"2024-04-28T12:37:27.977307+0000","last_fullsized":"2024-04-28T12:37:27.977307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840176+0000","last_change":"2024-04-28T12:37:26.867761+0000","last_active":"2024-04-28T12:37:27.840176+0000","last_peered":"2024-04-28T12:37:27.840176+0000","last_clean":"2024-04-28T12:37:27.840176+0000","last_became_active":"2024-04-28T12:37:26.867277+0000","last_became_peered":"2024-04-28T12:37:26.867277+0000","last_unstale":"2024-04-28T12:37:27.840176+0000","last_undegraded":"2024-04-28T12:37:27.840176+0000","last_fullsized":"2024-04-28T12:37:27.840176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:12:40.786592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977843+0000","last_change":"2024-04-28T12:37:21.812666+0000","last_active":"2024-04-28T12:37:27.977843+0000","last_peered":"2024-04-28T12:37:27.977843+0000","last_clean":"2024-04-28T12:37:27.977843+0000","last_became_active":"2024-04-28T12:37:21.812384+0000","last_became_peered":"2024-04-28T12:37:21.812384+0000","last_unstale":"2024-04-28T12:37:27.977843+0000","last_undegraded":"2024-04-28T12:37:27.977843+0000","last_fullsized":"2024-04-28T12:37:27.977843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:29:20.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844555+0000","last_change":"2024-04-28T12:37:22.794127+0000","last_active":"2024-04-28T12:37:27.844555+0000","last_peered":"2024-04-28T12:37:27.844555+0000","last_clean":"2024-04-28T12:37:27.844555+0000","last_became_active":"2024-04-28T12:37:22.793770+0000","last_became_peered":"2024-04-28T12:37:22.793770+0000","last_unstale":"2024-04-28T12:37:27.844555+0000","last_undegraded":"2024-04-28T12:37:27.844555+0000","last_fullsized":"2024-04-28T12:37:27.844555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:00:05.608247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977959+0000","last_change":"2024-04-28T12:37:26.863918+0000","last_active":"2024-04-28T12:37:27.977959+0000","last_peered":"2024-04-28T12:37:27.977959+0000","last_clean":"2024-04-28T12:37:27.977959+0000","last_became_active":"2024-04-28T12:37:26.863558+0000","last_became_peered":"2024-04-28T12:37:26.863558+0000","last_unstale":"2024-04-28T12:37:27.977959+0000","last_undegraded":"2024-04-28T12:37:27.977959+0000","last_fullsized":"2024-04-28T12:37:27.977959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:49:38.590277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.841795+0000","last_change":"2024-04-28T12:37:22.786731+0000","last_active":"2024-04-28T12:37:27.841795+0000","last_peered":"2024-04-28T12:37:27.841795+0000","last_clean":"2024-04-28T12:37:27.841795+0000","last_became_active":"2024-04-28T12:37:22.786283+0000","last_became_peered":"2024-04-28T12:37:22.786283+0000","last_unstale":"2024-04-28T12:37:27.841795+0000","last_undegraded":"2024-04-28T12:37:27.841795+0000","last_fullsized":"2024-04-28T12:37:27.841795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.838968+0000","last_change":"2024-04-28T12:37:22.793953+0000","last_active":"2024-04-28T12:37:27.838968+0000","last_peered":"2024-04-28T12:37:27.838968+0000","last_clean":"2024-04-28T12:37:27.838968+0000","last_became_active":"2024-04-28T12:37:22.793683+0000","last_became_peered":"2024-04-28T12:37:22.793683+0000","last_unstale":"2024-04-28T12:37:27.838968+0000","last_undegraded":"2024-04-28T12:37:27.838968+0000","last_fullsized":"2024-04-28T12:37:27.838968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:33.269962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978735+0000","last_change":"2024-04-28T12:37:22.791966+0000","last_active":"2024-04-28T12:37:27.978735+0000","last_peered":"2024-04-28T12:37:27.978735+0000","last_clean":"2024-04-28T12:37:27.978735+0000","last_became_active":"2024-04-28T12:37:22.791432+0000","last_became_peered":"2024-04-28T12:37:22.791432+0000","last_unstale":"2024-04-28T12:37:27.978735+0000","last_undegraded":"2024-04-28T12:37:27.978735+0000","last_fullsized":"2024-04-28T12:37:27.978735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:47.693882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977566+0000","last_change":"2024-04-28T12:37:22.786169+0000","last_active":"2024-04-28T12:37:27.977566+0000","last_peered":"2024-04-28T12:37:27.977566+0000","last_clean":"2024-04-28T12:37:27.977566+0000","last_became_active":"2024-04-28T12:37:22.785899+0000","last_became_peered":"2024-04-28T12:37:22.785899+0000","last_unstale":"2024-04-28T12:37:27.977566+0000","last_undegraded":"2024-04-28T12:37:27.977566+0000","last_fullsized":"2024-04-28T12:37:27.977566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:06:12.328002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979102+0000","last_change":"2024-04-28T12:37:22.812114+0000","last_active":"2024-04-28T12:37:27.979102+0000","last_peered":"2024-04-28T12:37:27.979102+0000","last_clean":"2024-04-28T12:37:27.979102+0000","last_became_active":"2024-04-28T12:37:22.808394+0000","last_became_peered":"2024-04-28T12:37:22.808394+0000","last_unstale":"2024-04-28T12:37:27.979102+0000","last_undegraded":"2024-04-28T12:37:27.979102+0000","last_fullsized":"2024-04-28T12:37:27.979102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976044+0000","last_change":"2024-04-28T12:37:22.803864+0000","last_active":"2024-04-28T12:37:27.976044+0000","last_peered":"2024-04-28T12:37:27.976044+0000","last_clean":"2024-04-28T12:37:27.976044+0000","last_became_active":"2024-04-28T12:37:22.803568+0000","last_became_peered":"2024-04-28T12:37:22.803568+0000","last_unstale":"2024-04-28T12:37:27.976044+0000","last_undegraded":"2024-04-28T12:37:27.976044+0000","last_fullsized":"2024-04-28T12:37:27.976044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844461+0000","last_change":"2024-04-28T12:37:22.788814+0000","last_active":"2024-04-28T12:37:27.844461+0000","last_peered":"2024-04-28T12:37:27.844461+0000","last_clean":"2024-04-28T12:37:27.844461+0000","last_became_active":"2024-04-28T12:37:22.786716+0000","last_became_peered":"2024-04-28T12:37:22.786716+0000","last_unstale":"2024-04-28T12:37:27.844461+0000","last_undegraded":"2024-04-28T12:37:27.844461+0000","last_fullsized":"2024-04-28T12:37:27.844461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978543+0000","last_change":"2024-04-28T12:37:22.812645+0000","last_active":"2024-04-28T12:37:27.978543+0000","last_peered":"2024-04-28T12:37:27.978543+0000","last_clean":"2024-04-28T12:37:27.978543+0000","last_became_active":"2024-04-28T12:37:22.811034+0000","last_became_peered":"2024-04-28T12:37:22.811034+0000","last_unstale":"2024-04-28T12:37:27.978543+0000","last_undegraded":"2024-04-28T12:37:27.978543+0000","last_fullsized":"2024-04-28T12:37:27.978543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978281+0000","last_change":"2024-04-28T12:37:22.804060+0000","last_active":"2024-04-28T12:37:27.978281+0000","last_peered":"2024-04-28T12:37:27.978281+0000","last_clean":"2024-04-28T12:37:27.978281+0000","last_became_active":"2024-04-28T12:37:22.803757+0000","last_became_peered":"2024-04-28T12:37:22.803757+0000","last_unstale":"2024-04-28T12:37:27.978281+0000","last_undegraded":"2024-04-28T12:37:27.978281+0000","last_fullsized":"2024-04-28T12:37:27.978281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977634+0000","last_change":"2024-04-28T12:37:22.795301+0000","last_active":"2024-04-28T12:37:27.977634+0000","last_peered":"2024-04-28T12:37:27.977634+0000","last_clean":"2024-04-28T12:37:27.977634+0000","last_became_active":"2024-04-28T12:37:22.794917+0000","last_became_peered":"2024-04-28T12:37:22.794917+0000","last_unstale":"2024-04-28T12:37:27.977634+0000","last_undegraded":"2024-04-28T12:37:27.977634+0000","last_fullsized":"2024-04-28T12:37:27.977634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:45.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978597+0000","last_change":"2024-04-28T12:37:22.805760+0000","last_active":"2024-04-28T12:37:27.978597+0000","last_peered":"2024-04-28T12:37:27.978597+0000","last_clean":"2024-04-28T12:37:27.978597+0000","last_became_active":"2024-04-28T12:37:22.805238+0000","last_became_peered":"2024-04-28T12:37:22.805238+0000","last_unstale":"2024-04-28T12:37:27.978597+0000","last_undegraded":"2024-04-28T12:37:27.978597+0000","last_fullsized":"2024-04-28T12:37:27.978597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:16.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976930+0000","last_change":"2024-04-28T12:37:22.793207+0000","last_active":"2024-04-28T12:37:27.976930+0000","last_peered":"2024-04-28T12:37:27.976930+0000","last_clean":"2024-04-28T12:37:27.976930+0000","last_became_active":"2024-04-28T12:37:22.792816+0000","last_became_peered":"2024-04-28T12:37:22.792816+0000","last_unstale":"2024-04-28T12:37:27.976930+0000","last_undegraded":"2024-04-28T12:37:27.976930+0000","last_fullsized":"2024-04-28T12:37:27.976930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:55:36.416184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842925+0000","last_change":"2024-04-28T12:37:22.780772+0000","last_active":"2024-04-28T12:37:27.842925+0000","last_peered":"2024-04-28T12:37:27.842925+0000","last_clean":"2024-04-28T12:37:27.842925+0000","last_became_active":"2024-04-28T12:37:22.780190+0000","last_became_peered":"2024-04-28T12:37:22.780190+0000","last_unstale":"2024-04-28T12:37:27.842925+0000","last_undegraded":"2024-04-28T12:37:27.842925+0000","last_fullsized":"2024-04-28T12:37:27.842925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982761+0000","last_change":"2024-04-28T12:37:22.799792+0000","last_active":"2024-04-28T12:37:27.982761+0000","last_peered":"2024-04-28T12:37:27.982761+0000","last_clean":"2024-04-28T12:37:27.982761+0000","last_became_active":"2024-04-28T12:37:22.799472+0000","last_became_peered":"2024-04-28T12:37:22.799472+0000","last_unstale":"2024-04-28T12:37:27.982761+0000","last_undegraded":"2024-04-28T12:37:27.982761+0000","last_fullsized":"2024-04-28T12:37:27.982761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:47.693882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840300+0000","last_change":"2024-04-28T12:37:22.793748+0000","last_active":"2024-04-28T12:37:27.840300+0000","last_peered":"2024-04-28T12:37:27.840300+0000","last_clean":"2024-04-28T12:37:27.840300+0000","last_became_active":"2024-04-28T12:37:22.793461+0000","last_became_peered":"2024-04-28T12:37:22.793461+0000","last_unstale":"2024-04-28T12:37:27.840300+0000","last_undegraded":"2024-04-28T12:37:27.840300+0000","last_fullsized":"2024-04-28T12:37:27.840300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:00:05.608247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980132+0000","last_change":"2024-04-28T12:37:22.816550+0000","last_active":"2024-04-28T12:37:27.980132+0000","last_peered":"2024-04-28T12:37:27.980132+0000","last_clean":"2024-04-28T12:37:27.980132+0000","last_became_active":"2024-04-28T12:37:22.811895+0000","last_became_peered":"2024-04-28T12:37:22.811895+0000","last_unstale":"2024-04-28T12:37:27.980132+0000","last_undegraded":"2024-04-28T12:37:27.980132+0000","last_fullsized":"2024-04-28T12:37:27.980132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:24.827452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977952+0000","last_change":"2024-04-28T12:37:22.803909+0000","last_active":"2024-04-28T12:37:27.977952+0000","last_peered":"2024-04-28T12:37:27.977952+0000","last_clean":"2024-04-28T12:37:27.977952+0000","last_became_active":"2024-04-28T12:37:22.803577+0000","last_became_peered":"2024-04-28T12:37:22.803577+0000","last_unstale":"2024-04-28T12:37:27.977952+0000","last_undegraded":"2024-04-28T12:37:27.977952+0000","last_fullsized":"2024-04-28T12:37:27.977952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:59.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.975883+0000","last_change":"2024-04-28T12:37:22.808602+0000","last_active":"2024-04-28T12:37:27.975883+0000","last_peered":"2024-04-28T12:37:27.975883+0000","last_clean":"2024-04-28T12:37:27.975883+0000","last_became_active":"2024-04-28T12:37:22.808300+0000","last_became_peered":"2024-04-28T12:37:22.808300+0000","last_unstale":"2024-04-28T12:37:27.975883+0000","last_undegraded":"2024-04-28T12:37:27.975883+0000","last_fullsized":"2024-04-28T12:37:27.975883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977024+0000","last_change":"2024-04-28T12:37:22.792556+0000","last_active":"2024-04-28T12:37:27.977024+0000","last_peered":"2024-04-28T12:37:27.977024+0000","last_clean":"2024-04-28T12:37:27.977024+0000","last_became_active":"2024-04-28T12:37:22.789568+0000","last_became_peered":"2024-04-28T12:37:22.789568+0000","last_unstale":"2024-04-28T12:37:27.977024+0000","last_undegraded":"2024-04-28T12:37:27.977024+0000","last_fullsized":"2024-04-28T12:37:27.977024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:12:36.729037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977547+0000","last_change":"2024-04-28T12:37:22.798420+0000","last_active":"2024-04-28T12:37:27.977547+0000","last_peered":"2024-04-28T12:37:27.977547+0000","last_clean":"2024-04-28T12:37:27.977547+0000","last_became_active":"2024-04-28T12:37:22.798125+0000","last_became_peered":"2024-04-28T12:37:22.798125+0000","last_unstale":"2024-04-28T12:37:27.977547+0000","last_undegraded":"2024-04-28T12:37:27.977547+0000","last_fullsized":"2024-04-28T12:37:27.977547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:42.893320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845776+0000","last_change":"2024-04-28T12:37:22.799783+0000","last_active":"2024-04-28T12:37:27.845776+0000","last_peered":"2024-04-28T12:37:27.845776+0000","last_clean":"2024-04-28T12:37:27.845776+0000","last_became_active":"2024-04-28T12:37:22.796468+0000","last_became_peered":"2024-04-28T12:37:22.796468+0000","last_unstale":"2024-04-28T12:37:27.845776+0000","last_undegraded":"2024-04-28T12:37:27.845776+0000","last_fullsized":"2024-04-28T12:37:27.845776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:00.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844178+0000","last_change":"2024-04-28T12:37:22.806029+0000","last_active":"2024-04-28T12:37:27.844178+0000","last_peered":"2024-04-28T12:37:27.844178+0000","last_clean":"2024-04-28T12:37:27.844178+0000","last_became_active":"2024-04-28T12:37:22.802600+0000","last_became_peered":"2024-04-28T12:37:22.802600+0000","last_unstale":"2024-04-28T12:37:27.844178+0000","last_undegraded":"2024-04-28T12:37:27.844178+0000","last_fullsized":"2024-04-28T12:37:27.844178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844198+0000","last_change":"2024-04-28T12:37:22.821112+0000","last_active":"2024-04-28T12:37:27.844198+0000","last_peered":"2024-04-28T12:37:27.844198+0000","last_clean":"2024-04-28T12:37:27.844198+0000","last_became_active":"2024-04-28T12:37:22.820777+0000","last_became_peered":"2024-04-28T12:37:22.820777+0000","last_unstale":"2024-04-28T12:37:27.844198+0000","last_undegraded":"2024-04-28T12:37:27.844198+0000","last_fullsized":"2024-04-28T12:37:27.844198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981775+0000","last_change":"2024-04-28T12:37:22.825673+0000","last_active":"2024-04-28T12:37:27.981775+0000","last_peered":"2024-04-28T12:37:27.981775+0000","last_clean":"2024-04-28T12:37:27.981775+0000","last_became_active":"2024-04-28T12:37:22.820991+0000","last_became_peered":"2024-04-28T12:37:22.820991+0000","last_unstale":"2024-04-28T12:37:27.981775+0000","last_undegraded":"2024-04-28T12:37:27.981775+0000","last_fullsized":"2024-04-28T12:37:27.981775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:00.502362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976734+0000","last_change":"2024-04-28T12:37:22.814076+0000","last_active":"2024-04-28T12:37:27.976734+0000","last_peered":"2024-04-28T12:37:27.976734+0000","last_clean":"2024-04-28T12:37:27.976734+0000","last_became_active":"2024-04-28T12:37:22.813739+0000","last_became_peered":"2024-04-28T12:37:22.813739+0000","last_unstale":"2024-04-28T12:37:27.976734+0000","last_undegraded":"2024-04-28T12:37:27.976734+0000","last_fullsized":"2024-04-28T12:37:27.976734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980558+0000","last_change":"2024-04-28T12:37:22.799783+0000","last_active":"2024-04-28T12:37:27.980558+0000","last_peered":"2024-04-28T12:37:27.980558+0000","last_clean":"2024-04-28T12:37:27.980558+0000","last_became_active":"2024-04-28T12:37:22.799519+0000","last_became_peered":"2024-04-28T12:37:22.799519+0000","last_unstale":"2024-04-28T12:37:27.980558+0000","last_undegraded":"2024-04-28T12:37:27.980558+0000","last_fullsized":"2024-04-28T12:37:27.980558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:59.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978456+0000","last_change":"2024-04-28T12:37:22.805539+0000","last_active":"2024-04-28T12:37:27.978456+0000","last_peered":"2024-04-28T12:37:27.978456+0000","last_clean":"2024-04-28T12:37:27.978456+0000","last_became_active":"2024-04-28T12:37:22.804834+0000","last_became_peered":"2024-04-28T12:37:22.804834+0000","last_unstale":"2024-04-28T12:37:27.978456+0000","last_undegraded":"2024-04-28T12:37:27.978456+0000","last_fullsized":"2024-04-28T12:37:27.978456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:47.618755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981101+0000","last_change":"2024-04-28T12:37:22.800776+0000","last_active":"2024-04-28T12:37:27.981101+0000","last_peered":"2024-04-28T12:37:27.981101+0000","last_clean":"2024-04-28T12:37:27.981101+0000","last_became_active":"2024-04-28T12:37:22.798852+0000","last_became_peered":"2024-04-28T12:37:22.798852+0000","last_unstale":"2024-04-28T12:37:27.981101+0000","last_undegraded":"2024-04-28T12:37:27.981101+0000","last_fullsized":"2024-04-28T12:37:27.981101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:24.827452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978903+0000","last_change":"2024-04-28T12:37:22.819772+0000","last_active":"2024-04-28T12:37:27.978903+0000","last_peered":"2024-04-28T12:37:27.978903+0000","last_clean":"2024-04-28T12:37:27.978903+0000","last_became_active":"2024-04-28T12:37:22.813375+0000","last_became_peered":"2024-04-28T12:37:22.813375+0000","last_unstale":"2024-04-28T12:37:27.978903+0000","last_undegraded":"2024-04-28T12:37:27.978903+0000","last_fullsized":"2024-04-28T12:37:27.978903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978010+0000","last_change":"2024-04-28T12:37:22.784463+0000","last_active":"2024-04-28T12:37:27.978010+0000","last_peered":"2024-04-28T12:37:27.978010+0000","last_clean":"2024-04-28T12:37:27.978010+0000","last_became_active":"2024-04-28T12:37:22.784183+0000","last_became_peered":"2024-04-28T12:37:22.784183+0000","last_unstale":"2024-04-28T12:37:27.978010+0000","last_undegraded":"2024-04-28T12:37:27.978010+0000","last_fullsized":"2024-04-28T12:37:27.978010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976443+0000","last_change":"2024-04-28T12:37:22.802869+0000","last_active":"2024-04-28T12:37:27.976443+0000","last_peered":"2024-04-28T12:37:27.976443+0000","last_clean":"2024-04-28T12:37:27.976443+0000","last_became_active":"2024-04-28T12:37:22.802598+0000","last_became_peered":"2024-04-28T12:37:22.802598+0000","last_unstale":"2024-04-28T12:37:27.976443+0000","last_undegraded":"2024-04-28T12:37:27.976443+0000","last_fullsized":"2024-04-28T12:37:27.976443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981568+0000","last_change":"2024-04-28T12:37:22.797900+0000","last_active":"2024-04-28T12:37:27.981568+0000","last_peered":"2024-04-28T12:37:27.981568+0000","last_clean":"2024-04-28T12:37:27.981568+0000","last_became_active":"2024-04-28T12:37:22.797533+0000","last_became_peered":"2024-04-28T12:37:22.797533+0000","last_unstale":"2024-04-28T12:37:27.981568+0000","last_undegraded":"2024-04-28T12:37:27.981568+0000","last_fullsized":"2024-04-28T12:37:27.981568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:08:46.974988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:37:52.485 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:37:52.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.726+0000 7faa73600640 1 Processor -- start 2024-04-28T12:37:52.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.726+0000 7faa73600640 1 -- start start 2024-04-28T12:37:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.728+0000 7faa73600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7faa6c1467c0 0x7faa6c072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.728+0000 7faa73600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c0b4140 0x7faa6c06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.728+0000 7faa73600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa6c146ba0 con 0x7faa6c0b3ce0 2024-04-28T12:37:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.728+0000 7faa73600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7faa6c073dd0 con 0x7faa6c1467c0 2024-04-28T12:37:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.728+0000 7faa73600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c0731c0 con 0x7faa6c0b4140 2024-04-28T12:37:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.728+0000 7faa70c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7faa6c1467c0 0x7faa6c072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.728+0000 7faa70c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7faa6c1467c0 0x7faa6c072b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45084/0 (socket says 172.21.15.131:45084) 2024-04-28T12:37:52.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.728+0000 7faa70c00640 1 -- 172.21.15.131:0/3632248483 learned_addr learned my addr 172.21.15.131:0/3632248483 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:52.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.728+0000 7faa71600640 1 --2- 172.21.15.131:0/3632248483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c0b4140 0x7faa6c06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:52.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.729+0000 7faa70c00640 1 -- 172.21.15.131:0/3632248483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c0b4140 msgr2=0x7faa6c06dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:52.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.729+0000 7faa70c00640 1 --2- 172.21.15.131:0/3632248483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c0b4140 0x7faa6c06dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.729+0000 7faa70c00640 1 -- 172.21.15.131:0/3632248483 >> v1:172.21.15.157:6789/0 conn(0x7faa6c0b3ce0 legacy=0x7faa6c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:52.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.729+0000 7faa70c00640 1 -- 172.21.15.131:0/3632248483 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c0734a0 con 0x7faa6c1467c0 2024-04-28T12:37:52.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.730+0000 7faa70c00640 1 --2- 172.21.15.131:0/3632248483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa6c1467c0 0x7faa6c072b70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faa6c073f50 tx=0x7faa5c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e9afc4f87ef0f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:52.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.730+0000 7faa6be00640 1 -- 172.21.15.131:0/3632248483 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c037070 con 0x7faa6c1467c0 2024-04-28T12:37:52.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.730+0000 7faa6be00640 1 -- 172.21.15.131:0/3632248483 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa5c032440 con 0x7faa6c1467c0 2024-04-28T12:37:52.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.730+0000 7faa6be00640 1 -- 172.21.15.131:0/3632248483 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c032810 con 0x7faa6c1467c0 2024-04-28T12:37:52.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.731+0000 7faa73600640 1 -- 172.21.15.131:0/3632248483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa6c1467c0 msgr2=0x7faa6c072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:52.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.731+0000 7faa73600640 1 --2- 172.21.15.131:0/3632248483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa6c1467c0 0x7faa6c072b70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faa6c073f50 tx=0x7faa5c02a250 comp rx=0 tx=0).stop 2024-04-28T12:37:52.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.731+0000 7faa73600640 1 -- 172.21.15.131:0/3632248483 shutdown_connections 2024-04-28T12:37:52.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.731+0000 7faa73600640 1 --2- 172.21.15.131:0/3632248483 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c0b4140 0x7faa6c06dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.731+0000 7faa73600640 1 --2- 172.21.15.131:0/3632248483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa6c1467c0 0x7faa6c072b70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.731+0000 7faa73600640 1 -- 172.21.15.131:0/3632248483 >> 172.21.15.131:0/3632248483 conn(0x7faa6c0b2900 msgr2=0x7faa6c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.731+0000 7faa73600640 1 -- 172.21.15.131:0/3632248483 shutdown_connections 2024-04-28T12:37:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.731+0000 7faa73600640 1 -- 172.21.15.131:0/3632248483 wait complete. 2024-04-28T12:37:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.733+0000 7faa73600640 1 Processor -- start 2024-04-28T12:37:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.733+0000 7faa73600640 1 -- start start 2024-04-28T12:37:52.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.734+0000 7faa73600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c0b3ce0 0x7faa6c1451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:52.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.734+0000 7faa73600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa6c1467c0 0x7faa6c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:52.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.734+0000 7faa73600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c160030 0x7faa6c0cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:52.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.734+0000 7faa73600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c0d1230 con 0x7faa6c1467c0 2024-04-28T12:37:52.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.734+0000 7faa73600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c073a60 con 0x7faa6c0b3ce0 2024-04-28T12:37:52.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.734+0000 7faa73600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c0d13b0 con 0x7faa6c160030 2024-04-28T12:37:52.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.734+0000 7faa63e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa6c1467c0 0x7faa6c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:52.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.734+0000 7faa63e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa6c1467c0 0x7faa6c15c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45088/0 (socket says 172.21.15.131:45088) 2024-04-28T12:37:52.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.734+0000 7faa63e00640 1 -- 172.21.15.131:0/477539822 learned_addr learned my addr 172.21.15.131:0/477539822 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:52.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.734+0000 7faa71600640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c160030 0x7faa6c0cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:52.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.735+0000 7faa70c00640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c0b3ce0 0x7faa6c1451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:52.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.735+0000 7faa63e00640 1 -- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c160030 msgr2=0x7faa6c0cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:52.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.735+0000 7faa63e00640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c160030 0x7faa6c0cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.735+0000 7faa63e00640 1 -- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c0b3ce0 msgr2=0x7faa6c1451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:52.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.735+0000 7faa63e00640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c0b3ce0 0x7faa6c1451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.735+0000 7faa63e00640 1 -- 172.21.15.131:0/477539822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c15dd80 con 0x7faa6c1467c0 2024-04-28T12:37:52.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.735+0000 7faa63e00640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa6c1467c0 0x7faa6c15c320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faa5400c400 tx=0x7faa5400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:52.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.735+0000 7faa69600640 1 -- 172.21.15.131:0/477539822 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa540052d0 con 0x7faa6c1467c0 2024-04-28T12:37:52.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.735+0000 7faa69600640 1 -- 172.21.15.131:0/477539822 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa54005490 con 0x7faa6c1467c0 2024-04-28T12:37:52.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.735+0000 7faa73600640 1 -- 172.21.15.131:0/477539822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c144990 con 0x7faa6c1467c0 2024-04-28T12:37:52.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.735+0000 7faa69600640 1 -- 172.21.15.131:0/477539822 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54007260 con 0x7faa6c1467c0 2024-04-28T12:37:52.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.735+0000 7faa73600640 1 -- 172.21.15.131:0/477539822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c0d0170 con 0x7faa6c1467c0 2024-04-28T12:37:52.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.737+0000 7faa73600640 1 -- 172.21.15.131:0/477539822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6c069350 con 0x7faa6c1467c0 2024-04-28T12:37:52.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.737+0000 7faa69600640 1 -- 172.21.15.131:0/477539822 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa54007400 con 0x7faa6c1467c0 2024-04-28T12:37:52.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.738+0000 7faa69600640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faa4806c8f0 0x7faa4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:52.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.738+0000 7faa69600640 1 -- 172.21.15.131:0/477539822 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faa5408b430 con 0x7faa6c1467c0 2024-04-28T12:37:52.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.739+0000 7faa70c00640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faa4806c8f0 0x7faa4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:52.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.740+0000 7faa70c00640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faa4806c8f0 0x7faa4806edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa5c034000 tx=0x7faa5c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:52.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.743+0000 7faa69600640 1 -- 172.21.15.131:0/477539822 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa54059510 con 0x7faa6c1467c0 2024-04-28T12:37:52.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.940+0000 7faa73600640 1 -- 172.21.15.131:0/477539822 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa6c15c860 con 0x7faa4806c8f0 2024-04-28T12:37:52.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.964+0000 7faa69600640 1 -- 172.21.15.131:0/477539822 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7faa5c032730 con 0x7faa4806c8f0 2024-04-28T12:37:52.964 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:52.968 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:37:52.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.971+0000 7faa73600640 1 -- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faa4806c8f0 msgr2=0x7faa4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:52.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.971+0000 7faa73600640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faa4806c8f0 0x7faa4806edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa5c034000 tx=0x7faa5c02e000 comp rx=0 tx=0).stop 2024-04-28T12:37:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.971+0000 7faa73600640 1 -- 172.21.15.131:0/477539822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa6c1467c0 msgr2=0x7faa6c15c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.971+0000 7faa73600640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa6c1467c0 0x7faa6c15c320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faa5400c400 tx=0x7faa5400c8c0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.971+0000 7faa73600640 1 -- 172.21.15.131:0/477539822 shutdown_connections 2024-04-28T12:37:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.971+0000 7faa73600640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faa4806c8f0 0x7faa4806edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.972+0000 7faa73600640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa6c160030 0x7faa6c0cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.972+0000 7faa73600640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa6c1467c0 0x7faa6c15c320 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.972+0000 7faa73600640 1 --2- 172.21.15.131:0/477539822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa6c0b3ce0 0x7faa6c1451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.972+0000 7faa73600640 1 -- 172.21.15.131:0/477539822 >> 172.21.15.131:0/477539822 conn(0x7faa6c0b2900 msgr2=0x7faa6c0d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.972+0000 7faa73600640 1 -- 172.21.15.131:0/477539822 shutdown_connections 2024-04-28T12:37:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:52.972+0000 7faa73600640 1 -- 172.21.15.131:0/477539822 wait complete. 2024-04-28T12:37:52.988 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-28T12:37:52.925549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231864,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742856,"statfs":{"total":773094113280,"available":772856684544,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005775"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980293+0000","last_change":"2024-04-28T12:37:21.812889+0000","last_active":"2024-04-28T12:37:27.980293+0000","last_peered":"2024-04-28T12:37:27.980293+0000","last_clean":"2024-04-28T12:37:27.980293+0000","last_became_active":"2024-04-28T12:37:21.812593+0000","last_became_peered":"2024-04-28T12:37:21.812593+0000","last_unstale":"2024-04-28T12:37:27.980293+0000","last_undegraded":"2024-04-28T12:37:27.980293+0000","last_fullsized":"2024-04-28T12:37:27.980293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:33.616030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978052+0000","last_change":"2024-04-28T12:37:22.794799+0000","last_active":"2024-04-28T12:37:27.978052+0000","last_peered":"2024-04-28T12:37:27.978052+0000","last_clean":"2024-04-28T12:37:27.978052+0000","last_became_active":"2024-04-28T12:37:22.794480+0000","last_became_peered":"2024-04-28T12:37:22.794480+0000","last_unstale":"2024-04-28T12:37:27.978052+0000","last_undegraded":"2024-04-28T12:37:27.978052+0000","last_fullsized":"2024-04-28T12:37:27.978052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:39.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976789+0000","last_change":"2024-04-28T12:37:26.883025+0000","last_active":"2024-04-28T12:37:27.976789+0000","last_peered":"2024-04-28T12:37:27.976789+0000","last_clean":"2024-04-28T12:37:27.976789+0000","last_became_active":"2024-04-28T12:37:26.882577+0000","last_became_peered":"2024-04-28T12:37:26.882577+0000","last_unstale":"2024-04-28T12:37:27.976789+0000","last_undegraded":"2024-04-28T12:37:27.976789+0000","last_fullsized":"2024-04-28T12:37:27.976789+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:09.634939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978274+0000","last_change":"2024-04-28T12:37:21.796325+0000","last_active":"2024-04-28T12:37:27.978274+0000","last_peered":"2024-04-28T12:37:27.978274+0000","last_clean":"2024-04-28T12:37:27.978274+0000","last_became_active":"2024-04-28T12:37:21.795878+0000","last_became_peered":"2024-04-28T12:37:21.795878+0000","last_unstale":"2024-04-28T12:37:27.978274+0000","last_undegraded":"2024-04-28T12:37:27.978274+0000","last_fullsized":"2024-04-28T12:37:27.978274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:55:28.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979406+0000","last_change":"2024-04-28T12:37:22.821287+0000","last_active":"2024-04-28T12:37:27.979406+0000","last_peered":"2024-04-28T12:37:27.979406+0000","last_clean":"2024-04-28T12:37:27.979406+0000","last_became_active":"2024-04-28T12:37:22.820999+0000","last_became_peered":"2024-04-28T12:37:22.820999+0000","last_unstale":"2024-04-28T12:37:27.979406+0000","last_undegraded":"2024-04-28T12:37:27.979406+0000","last_fullsized":"2024-04-28T12:37:27.979406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843680+0000","last_change":"2024-04-28T12:37:26.860456+0000","last_active":"2024-04-28T12:37:27.843680+0000","last_peered":"2024-04-28T12:37:27.843680+0000","last_clean":"2024-04-28T12:37:27.843680+0000","last_became_active":"2024-04-28T12:37:26.859360+0000","last_became_peered":"2024-04-28T12:37:26.859360+0000","last_unstale":"2024-04-28T12:37:27.843680+0000","last_undegraded":"2024-04-28T12:37:27.843680+0000","last_fullsized":"2024-04-28T12:37:27.843680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:17.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982655+0000","last_change":"2024-04-28T12:37:21.817405+0000","last_active":"2024-04-28T12:37:27.982655+0000","last_peered":"2024-04-28T12:37:27.982655+0000","last_clean":"2024-04-28T12:37:27.982655+0000","last_became_active":"2024-04-28T12:37:21.816935+0000","last_became_peered":"2024-04-28T12:37:21.816935+0000","last_unstale":"2024-04-28T12:37:27.982655+0000","last_undegraded":"2024-04-28T12:37:27.982655+0000","last_fullsized":"2024-04-28T12:37:27.982655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:32.253270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978974+0000","last_change":"2024-04-28T12:37:22.809893+0000","last_active":"2024-04-28T12:37:27.978974+0000","last_peered":"2024-04-28T12:37:27.978974+0000","last_clean":"2024-04-28T12:37:27.978974+0000","last_became_active":"2024-04-28T12:37:22.808912+0000","last_became_peered":"2024-04-28T12:37:22.808912+0000","last_unstale":"2024-04-28T12:37:27.978974+0000","last_undegraded":"2024-04-28T12:37:27.978974+0000","last_fullsized":"2024-04-28T12:37:27.978974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:39.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843302+0000","last_change":"2024-04-28T12:37:26.870945+0000","last_active":"2024-04-28T12:37:27.843302+0000","last_peered":"2024-04-28T12:37:27.843302+0000","last_clean":"2024-04-28T12:37:27.843302+0000","last_became_active":"2024-04-28T12:37:26.867901+0000","last_became_peered":"2024-04-28T12:37:26.867901+0000","last_unstale":"2024-04-28T12:37:27.843302+0000","last_undegraded":"2024-04-28T12:37:27.843302+0000","last_fullsized":"2024-04-28T12:37:27.843302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:50.835488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981355+0000","last_change":"2024-04-28T12:37:21.816630+0000","last_active":"2024-04-28T12:37:27.981355+0000","last_peered":"2024-04-28T12:37:27.981355+0000","last_clean":"2024-04-28T12:37:27.981355+0000","last_became_active":"2024-04-28T12:37:21.815735+0000","last_became_peered":"2024-04-28T12:37:21.815735+0000","last_unstale":"2024-04-28T12:37:27.981355+0000","last_undegraded":"2024-04-28T12:37:27.981355+0000","last_fullsized":"2024-04-28T12:37:27.981355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:46:26.715438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979607+0000","last_change":"2024-04-28T12:37:22.810382+0000","last_active":"2024-04-28T12:37:27.979607+0000","last_peered":"2024-04-28T12:37:27.979607+0000","last_clean":"2024-04-28T12:37:27.979607+0000","last_became_active":"2024-04-28T12:37:22.809175+0000","last_became_peered":"2024-04-28T12:37:22.809175+0000","last_unstale":"2024-04-28T12:37:27.979607+0000","last_undegraded":"2024-04-28T12:37:27.979607+0000","last_fullsized":"2024-04-28T12:37:27.979607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:45.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844410+0000","last_change":"2024-04-28T12:37:26.864046+0000","last_active":"2024-04-28T12:37:27.844410+0000","last_peered":"2024-04-28T12:37:27.844410+0000","last_clean":"2024-04-28T12:37:27.844410+0000","last_became_active":"2024-04-28T12:37:26.863585+0000","last_became_peered":"2024-04-28T12:37:26.863585+0000","last_unstale":"2024-04-28T12:37:27.844410+0000","last_undegraded":"2024-04-28T12:37:27.844410+0000","last_fullsized":"2024-04-28T12:37:27.844410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:26:03.360606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842825+0000","last_change":"2024-04-28T12:37:22.794854+0000","last_active":"2024-04-28T12:37:27.842825+0000","last_peered":"2024-04-28T12:37:27.842825+0000","last_clean":"2024-04-28T12:37:27.842825+0000","last_became_active":"2024-04-28T12:37:22.794522+0000","last_became_peered":"2024-04-28T12:37:22.794522+0000","last_unstale":"2024-04-28T12:37:27.842825+0000","last_undegraded":"2024-04-28T12:37:27.842825+0000","last_fullsized":"2024-04-28T12:37:27.842825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981814+0000","last_change":"2024-04-28T12:37:21.814514+0000","last_active":"2024-04-28T12:37:27.981814+0000","last_peered":"2024-04-28T12:37:27.981814+0000","last_clean":"2024-04-28T12:37:27.981814+0000","last_became_active":"2024-04-28T12:37:21.814226+0000","last_became_peered":"2024-04-28T12:37:21.814226+0000","last_unstale":"2024-04-28T12:37:27.981814+0000","last_undegraded":"2024-04-28T12:37:27.981814+0000","last_fullsized":"2024-04-28T12:37:27.981814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.846440+0000","last_change":"2024-04-28T12:37:26.869683+0000","last_active":"2024-04-28T12:37:27.846440+0000","last_peered":"2024-04-28T12:37:27.846440+0000","last_clean":"2024-04-28T12:37:27.846440+0000","last_became_active":"2024-04-28T12:37:26.869160+0000","last_became_peered":"2024-04-28T12:37:26.869160+0000","last_unstale":"2024-04-28T12:37:27.846440+0000","last_undegraded":"2024-04-28T12:37:27.846440+0000","last_fullsized":"2024-04-28T12:37:27.846440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977688+0000","last_change":"2024-04-28T12:37:22.797734+0000","last_active":"2024-04-28T12:37:27.977688+0000","last_peered":"2024-04-28T12:37:27.977688+0000","last_clean":"2024-04-28T12:37:27.977688+0000","last_became_active":"2024-04-28T12:37:22.797443+0000","last_became_peered":"2024-04-28T12:37:22.797443+0000","last_unstale":"2024-04-28T12:37:27.977688+0000","last_undegraded":"2024-04-28T12:37:27.977688+0000","last_fullsized":"2024-04-28T12:37:27.977688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978047+0000","last_change":"2024-04-28T12:37:21.798038+0000","last_active":"2024-04-28T12:37:27.978047+0000","last_peered":"2024-04-28T12:37:27.978047+0000","last_clean":"2024-04-28T12:37:27.978047+0000","last_became_active":"2024-04-28T12:37:21.797779+0000","last_became_peered":"2024-04-28T12:37:21.797779+0000","last_unstale":"2024-04-28T12:37:27.978047+0000","last_undegraded":"2024-04-28T12:37:27.978047+0000","last_fullsized":"2024-04-28T12:37:27.978047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:32.253270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845656+0000","last_change":"2024-04-28T12:37:26.869060+0000","last_active":"2024-04-28T12:37:27.845656+0000","last_peered":"2024-04-28T12:37:27.845656+0000","last_clean":"2024-04-28T12:37:27.845656+0000","last_became_active":"2024-04-28T12:37:26.867405+0000","last_became_peered":"2024-04-28T12:37:26.867405+0000","last_unstale":"2024-04-28T12:37:27.845656+0000","last_undegraded":"2024-04-28T12:37:27.845656+0000","last_fullsized":"2024-04-28T12:37:27.845656+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:29:29.617319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977155+0000","last_change":"2024-04-28T12:37:22.789718+0000","last_active":"2024-04-28T12:37:27.977155+0000","last_peered":"2024-04-28T12:37:27.977155+0000","last_clean":"2024-04-28T12:37:27.977155+0000","last_became_active":"2024-04-28T12:37:22.786954+0000","last_became_peered":"2024-04-28T12:37:22.786954+0000","last_unstale":"2024-04-28T12:37:27.977155+0000","last_undegraded":"2024-04-28T12:37:27.977155+0000","last_fullsized":"2024-04-28T12:37:27.977155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.975988+0000","last_change":"2024-04-28T12:37:21.804262+0000","last_active":"2024-04-28T12:37:27.975988+0000","last_peered":"2024-04-28T12:37:27.975988+0000","last_clean":"2024-04-28T12:37:27.975988+0000","last_became_active":"2024-04-28T12:37:21.803851+0000","last_became_peered":"2024-04-28T12:37:21.803851+0000","last_unstale":"2024-04-28T12:37:27.975988+0000","last_undegraded":"2024-04-28T12:37:27.975988+0000","last_fullsized":"2024-04-28T12:37:27.975988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:39.334819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978307+0000","last_change":"2024-04-28T12:37:26.884595+0000","last_active":"2024-04-28T12:37:27.978307+0000","last_peered":"2024-04-28T12:37:27.978307+0000","last_clean":"2024-04-28T12:37:27.978307+0000","last_became_active":"2024-04-28T12:37:26.883799+0000","last_became_peered":"2024-04-28T12:37:26.883799+0000","last_unstale":"2024-04-28T12:37:27.978307+0000","last_undegraded":"2024-04-28T12:37:27.978307+0000","last_fullsized":"2024-04-28T12:37:27.978307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:28.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844830+0000","last_change":"2024-04-28T12:37:22.778971+0000","last_active":"2024-04-28T12:37:27.844830+0000","last_peered":"2024-04-28T12:37:27.844830+0000","last_clean":"2024-04-28T12:37:27.844830+0000","last_became_active":"2024-04-28T12:37:22.778674+0000","last_became_peered":"2024-04-28T12:37:22.778674+0000","last_unstale":"2024-04-28T12:37:27.844830+0000","last_undegraded":"2024-04-28T12:37:27.844830+0000","last_fullsized":"2024-04-28T12:37:27.844830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:54.666754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979280+0000","last_change":"2024-04-28T12:37:21.816284+0000","last_active":"2024-04-28T12:37:27.979280+0000","last_peered":"2024-04-28T12:37:27.979280+0000","last_clean":"2024-04-28T12:37:27.979280+0000","last_became_active":"2024-04-28T12:37:21.815140+0000","last_became_peered":"2024-04-28T12:37:21.815140+0000","last_unstale":"2024-04-28T12:37:27.979280+0000","last_undegraded":"2024-04-28T12:37:27.979280+0000","last_fullsized":"2024-04-28T12:37:27.979280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:53.650061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.846043+0000","last_change":"2024-04-28T12:37:26.868136+0000","last_active":"2024-04-28T12:37:27.846043+0000","last_peered":"2024-04-28T12:37:27.846043+0000","last_clean":"2024-04-28T12:37:27.846043+0000","last_became_active":"2024-04-28T12:37:26.867663+0000","last_became_peered":"2024-04-28T12:37:26.867663+0000","last_unstale":"2024-04-28T12:37:27.846043+0000","last_undegraded":"2024-04-28T12:37:27.846043+0000","last_fullsized":"2024-04-28T12:37:27.846043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:45.428951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978823+0000","last_change":"2024-04-28T12:37:22.797968+0000","last_active":"2024-04-28T12:37:27.978823+0000","last_peered":"2024-04-28T12:37:27.978823+0000","last_clean":"2024-04-28T12:37:27.978823+0000","last_became_active":"2024-04-28T12:37:22.797258+0000","last_became_peered":"2024-04-28T12:37:22.797258+0000","last_unstale":"2024-04-28T12:37:27.978823+0000","last_undegraded":"2024-04-28T12:37:27.978823+0000","last_fullsized":"2024-04-28T12:37:27.978823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843883+0000","last_change":"2024-04-28T12:37:21.814135+0000","last_active":"2024-04-28T12:37:27.843883+0000","last_peered":"2024-04-28T12:37:27.843883+0000","last_clean":"2024-04-28T12:37:27.843883+0000","last_became_active":"2024-04-28T12:37:21.813800+0000","last_became_peered":"2024-04-28T12:37:21.813800+0000","last_unstale":"2024-04-28T12:37:27.843883+0000","last_undegraded":"2024-04-28T12:37:27.843883+0000","last_fullsized":"2024-04-28T12:37:27.843883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:49.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844709+0000","last_change":"2024-04-28T12:37:26.885109+0000","last_active":"2024-04-28T12:37:27.844709+0000","last_peered":"2024-04-28T12:37:27.844709+0000","last_clean":"2024-04-28T12:37:27.844709+0000","last_became_active":"2024-04-28T12:37:26.882922+0000","last_became_peered":"2024-04-28T12:37:26.882922+0000","last_unstale":"2024-04-28T12:37:27.844709+0000","last_undegraded":"2024-04-28T12:37:27.844709+0000","last_fullsized":"2024-04-28T12:37:27.844709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:20.147514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978601+0000","last_change":"2024-04-28T12:37:22.802331+0000","last_active":"2024-04-28T12:37:27.978601+0000","last_peered":"2024-04-28T12:37:27.978601+0000","last_clean":"2024-04-28T12:37:27.978601+0000","last_became_active":"2024-04-28T12:37:22.801939+0000","last_became_peered":"2024-04-28T12:37:22.801939+0000","last_unstale":"2024-04-28T12:37:27.978601+0000","last_undegraded":"2024-04-28T12:37:27.978601+0000","last_fullsized":"2024-04-28T12:37:27.978601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:59.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976553+0000","last_change":"2024-04-28T12:37:21.796110+0000","last_active":"2024-04-28T12:37:27.976553+0000","last_peered":"2024-04-28T12:37:27.976553+0000","last_clean":"2024-04-28T12:37:27.976553+0000","last_became_active":"2024-04-28T12:37:21.795405+0000","last_became_peered":"2024-04-28T12:37:21.795405+0000","last_unstale":"2024-04-28T12:37:27.976553+0000","last_undegraded":"2024-04-28T12:37:27.976553+0000","last_fullsized":"2024-04-28T12:37:27.976553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:53.650061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976613+0000","last_change":"2024-04-28T12:37:26.883160+0000","last_active":"2024-04-28T12:37:27.976613+0000","last_peered":"2024-04-28T12:37:27.976613+0000","last_clean":"2024-04-28T12:37:27.976613+0000","last_became_active":"2024-04-28T12:37:26.882684+0000","last_became_peered":"2024-04-28T12:37:26.882684+0000","last_unstale":"2024-04-28T12:37:27.976613+0000","last_undegraded":"2024-04-28T12:37:27.976613+0000","last_fullsized":"2024-04-28T12:37:27.976613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:19:01.590267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981129+0000","last_change":"2024-04-28T12:37:22.800581+0000","last_active":"2024-04-28T12:37:27.981129+0000","last_peered":"2024-04-28T12:37:27.981129+0000","last_clean":"2024-04-28T12:37:27.981129+0000","last_became_active":"2024-04-28T12:37:22.800241+0000","last_became_peered":"2024-04-28T12:37:22.800241+0000","last_unstale":"2024-04-28T12:37:27.981129+0000","last_undegraded":"2024-04-28T12:37:27.981129+0000","last_fullsized":"2024-04-28T12:37:27.981129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982900+0000","last_change":"2024-04-28T12:37:21.811668+0000","last_active":"2024-04-28T12:37:27.982900+0000","last_peered":"2024-04-28T12:37:27.982900+0000","last_clean":"2024-04-28T12:37:27.982900+0000","last_became_active":"2024-04-28T12:37:21.811372+0000","last_became_peered":"2024-04-28T12:37:21.811372+0000","last_unstale":"2024-04-28T12:37:27.982900+0000","last_undegraded":"2024-04-28T12:37:27.982900+0000","last_fullsized":"2024-04-28T12:37:27.982900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:00:04.591555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976863+0000","last_change":"2024-04-28T12:37:26.877753+0000","last_active":"2024-04-28T12:37:27.976863+0000","last_peered":"2024-04-28T12:37:27.976863+0000","last_clean":"2024-04-28T12:37:27.976863+0000","last_became_active":"2024-04-28T12:37:26.877456+0000","last_became_peered":"2024-04-28T12:37:26.877456+0000","last_unstale":"2024-04-28T12:37:27.976863+0000","last_undegraded":"2024-04-28T12:37:27.976863+0000","last_fullsized":"2024-04-28T12:37:27.976863+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:45.428951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979882+0000","last_change":"2024-04-28T12:37:22.801683+0000","last_active":"2024-04-28T12:37:27.979882+0000","last_peered":"2024-04-28T12:37:27.979882+0000","last_clean":"2024-04-28T12:37:27.979882+0000","last_became_active":"2024-04-28T12:37:22.801378+0000","last_became_peered":"2024-04-28T12:37:22.801378+0000","last_unstale":"2024-04-28T12:37:27.979882+0000","last_undegraded":"2024-04-28T12:37:27.979882+0000","last_fullsized":"2024-04-28T12:37:27.979882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:41.371396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981252+0000","last_change":"2024-04-28T12:37:21.791074+0000","last_active":"2024-04-28T12:37:27.981252+0000","last_peered":"2024-04-28T12:37:27.981252+0000","last_clean":"2024-04-28T12:37:27.981252+0000","last_became_active":"2024-04-28T12:37:21.790645+0000","last_became_peered":"2024-04-28T12:37:21.790645+0000","last_unstale":"2024-04-28T12:37:27.981252+0000","last_undegraded":"2024-04-28T12:37:27.981252+0000","last_fullsized":"2024-04-28T12:37:27.981252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977495+0000","last_change":"2024-04-28T12:37:26.873652+0000","last_active":"2024-04-28T12:37:27.977495+0000","last_peered":"2024-04-28T12:37:27.977495+0000","last_clean":"2024-04-28T12:37:27.977495+0000","last_became_active":"2024-04-28T12:37:26.873065+0000","last_became_peered":"2024-04-28T12:37:26.873065+0000","last_unstale":"2024-04-28T12:37:27.977495+0000","last_undegraded":"2024-04-28T12:37:27.977495+0000","last_fullsized":"2024-04-28T12:37:27.977495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845437+0000","last_change":"2024-04-28T12:37:22.794106+0000","last_active":"2024-04-28T12:37:27.845437+0000","last_peered":"2024-04-28T12:37:27.845437+0000","last_clean":"2024-04-28T12:37:27.845437+0000","last_became_active":"2024-04-28T12:37:22.789651+0000","last_became_peered":"2024-04-28T12:37:22.789651+0000","last_unstale":"2024-04-28T12:37:27.845437+0000","last_undegraded":"2024-04-28T12:37:27.845437+0000","last_fullsized":"2024-04-28T12:37:27.845437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:33.269962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979186+0000","last_change":"2024-04-28T12:37:21.810116+0000","last_active":"2024-04-28T12:37:27.979186+0000","last_peered":"2024-04-28T12:37:27.979186+0000","last_clean":"2024-04-28T12:37:27.979186+0000","last_became_active":"2024-04-28T12:37:21.809736+0000","last_became_peered":"2024-04-28T12:37:21.809736+0000","last_unstale":"2024-04-28T12:37:27.979186+0000","last_undegraded":"2024-04-28T12:37:27.979186+0000","last_fullsized":"2024-04-28T12:37:27.979186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:39.334819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981752+0000","last_change":"2024-04-28T12:37:26.867239+0000","last_active":"2024-04-28T12:37:27.981752+0000","last_peered":"2024-04-28T12:37:27.981752+0000","last_clean":"2024-04-28T12:37:27.981752+0000","last_became_active":"2024-04-28T12:37:26.866848+0000","last_became_peered":"2024-04-28T12:37:26.866848+0000","last_unstale":"2024-04-28T12:37:27.981752+0000","last_undegraded":"2024-04-28T12:37:27.981752+0000","last_fullsized":"2024-04-28T12:37:27.981752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:08:51.032543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976301+0000","last_change":"2024-04-28T12:37:22.796512+0000","last_active":"2024-04-28T12:37:27.976301+0000","last_peered":"2024-04-28T12:37:27.976301+0000","last_clean":"2024-04-28T12:37:27.976301+0000","last_became_active":"2024-04-28T12:37:22.793368+0000","last_became_peered":"2024-04-28T12:37:22.793368+0000","last_unstale":"2024-04-28T12:37:27.976301+0000","last_undegraded":"2024-04-28T12:37:27.976301+0000","last_fullsized":"2024-04-28T12:37:27.976301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.839137+0000","last_change":"2024-04-28T12:37:21.793644+0000","last_active":"2024-04-28T12:37:27.839137+0000","last_peered":"2024-04-28T12:37:27.839137+0000","last_clean":"2024-04-28T12:37:27.839137+0000","last_became_active":"2024-04-28T12:37:21.793358+0000","last_became_peered":"2024-04-28T12:37:21.793358+0000","last_unstale":"2024-04-28T12:37:27.839137+0000","last_undegraded":"2024-04-28T12:37:27.839137+0000","last_fullsized":"2024-04-28T12:37:27.839137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:55:28.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978474+0000","last_change":"2024-04-28T12:37:26.866571+0000","last_active":"2024-04-28T12:37:27.978474+0000","last_peered":"2024-04-28T12:37:27.978474+0000","last_clean":"2024-04-28T12:37:27.978474+0000","last_became_active":"2024-04-28T12:37:26.866042+0000","last_became_peered":"2024-04-28T12:37:26.866042+0000","last_unstale":"2024-04-28T12:37:27.978474+0000","last_undegraded":"2024-04-28T12:37:27.978474+0000","last_fullsized":"2024-04-28T12:37:27.978474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:04.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976223+0000","last_change":"2024-04-28T12:37:22.796747+0000","last_active":"2024-04-28T12:37:27.976223+0000","last_peered":"2024-04-28T12:37:27.976223+0000","last_clean":"2024-04-28T12:37:27.976223+0000","last_became_active":"2024-04-28T12:37:22.795677+0000","last_became_peered":"2024-04-28T12:37:22.795677+0000","last_unstale":"2024-04-28T12:37:27.976223+0000","last_undegraded":"2024-04-28T12:37:27.976223+0000","last_fullsized":"2024-04-28T12:37:27.976223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:05:18.644675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979595+0000","last_change":"2024-04-28T12:37:21.814725+0000","last_active":"2024-04-28T12:37:27.979595+0000","last_peered":"2024-04-28T12:37:27.979595+0000","last_clean":"2024-04-28T12:37:27.979595+0000","last_became_active":"2024-04-28T12:37:21.814516+0000","last_became_peered":"2024-04-28T12:37:21.814516+0000","last_unstale":"2024-04-28T12:37:27.979595+0000","last_undegraded":"2024-04-28T12:37:27.979595+0000","last_fullsized":"2024-04-28T12:37:27.979595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:49.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978718+0000","last_change":"2024-04-28T12:37:26.886450+0000","last_active":"2024-04-28T12:37:27.978718+0000","last_peered":"2024-04-28T12:37:27.978718+0000","last_clean":"2024-04-28T12:37:27.978718+0000","last_became_active":"2024-04-28T12:37:26.883398+0000","last_became_peered":"2024-04-28T12:37:26.883398+0000","last_unstale":"2024-04-28T12:37:27.978718+0000","last_undegraded":"2024-04-28T12:37:27.978718+0000","last_fullsized":"2024-04-28T12:37:27.978718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:04.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977534+0000","last_change":"2024-04-28T12:37:22.814159+0000","last_active":"2024-04-28T12:37:27.977534+0000","last_peered":"2024-04-28T12:37:27.977534+0000","last_clean":"2024-04-28T12:37:27.977534+0000","last_became_active":"2024-04-28T12:37:22.813889+0000","last_became_peered":"2024-04-28T12:37:22.813889+0000","last_unstale":"2024-04-28T12:37:27.977534+0000","last_undegraded":"2024-04-28T12:37:27.977534+0000","last_fullsized":"2024-04-28T12:37:27.977534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:16.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979063+0000","last_change":"2024-04-28T12:37:21.803169+0000","last_active":"2024-04-28T12:37:27.979063+0000","last_peered":"2024-04-28T12:37:27.979063+0000","last_clean":"2024-04-28T12:37:27.979063+0000","last_became_active":"2024-04-28T12:37:21.800812+0000","last_became_peered":"2024-04-28T12:37:21.800812+0000","last_unstale":"2024-04-28T12:37:27.979063+0000","last_undegraded":"2024-04-28T12:37:27.979063+0000","last_fullsized":"2024-04-28T12:37:27.979063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:55:28.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979702+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:37:27.979702+0000","last_peered":"2024-04-28T12:37:27.979702+0000","last_clean":"2024-04-28T12:37:27.979702+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:37:27.979702+0000","last_undegraded":"2024-04-28T12:37:27.979702+0000","last_fullsized":"2024-04-28T12:37:27.979702+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843882+0000","last_change":"2024-04-28T12:37:26.884731+0000","last_active":"2024-04-28T12:37:27.843882+0000","last_peered":"2024-04-28T12:37:27.843882+0000","last_clean":"2024-04-28T12:37:27.843882+0000","last_became_active":"2024-04-28T12:37:26.882799+0000","last_became_peered":"2024-04-28T12:37:26.882799+0000","last_unstale":"2024-04-28T12:37:27.843882+0000","last_undegraded":"2024-04-28T12:37:27.843882+0000","last_fullsized":"2024-04-28T12:37:27.843882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:51.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982421+0000","last_change":"2024-04-28T12:37:22.818975+0000","last_active":"2024-04-28T12:37:27.982421+0000","last_peered":"2024-04-28T12:37:27.982421+0000","last_clean":"2024-04-28T12:37:27.982421+0000","last_became_active":"2024-04-28T12:37:22.818677+0000","last_became_peered":"2024-04-28T12:37:22.818677+0000","last_unstale":"2024-04-28T12:37:27.982421+0000","last_undegraded":"2024-04-28T12:37:27.982421+0000","last_fullsized":"2024-04-28T12:37:27.982421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:41.371396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840221+0000","last_change":"2024-04-28T12:37:21.800771+0000","last_active":"2024-04-28T12:37:27.840221+0000","last_peered":"2024-04-28T12:37:27.840221+0000","last_clean":"2024-04-28T12:37:27.840221+0000","last_became_active":"2024-04-28T12:37:21.800451+0000","last_became_peered":"2024-04-28T12:37:21.800451+0000","last_unstale":"2024-04-28T12:37:27.840221+0000","last_undegraded":"2024-04-28T12:37:27.840221+0000","last_fullsized":"2024-04-28T12:37:27.840221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842666+0000","last_change":"2024-04-28T12:37:26.879284+0000","last_active":"2024-04-28T12:37:27.842666+0000","last_peered":"2024-04-28T12:37:27.842666+0000","last_clean":"2024-04-28T12:37:27.842666+0000","last_became_active":"2024-04-28T12:37:26.878864+0000","last_became_peered":"2024-04-28T12:37:26.878864+0000","last_unstale":"2024-04-28T12:37:27.842666+0000","last_undegraded":"2024-04-28T12:37:27.842666+0000","last_fullsized":"2024-04-28T12:37:27.842666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:45.428951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979686+0000","last_change":"2024-04-28T12:37:22.824563+0000","last_active":"2024-04-28T12:37:27.979686+0000","last_peered":"2024-04-28T12:37:27.979686+0000","last_clean":"2024-04-28T12:37:27.979686+0000","last_became_active":"2024-04-28T12:37:22.818493+0000","last_became_peered":"2024-04-28T12:37:22.818493+0000","last_unstale":"2024-04-28T12:37:27.979686+0000","last_undegraded":"2024-04-28T12:37:27.979686+0000","last_fullsized":"2024-04-28T12:37:27.979686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:00.502362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976419+0000","last_change":"2024-04-28T12:37:21.799906+0000","last_active":"2024-04-28T12:37:27.976419+0000","last_peered":"2024-04-28T12:37:27.976419+0000","last_clean":"2024-04-28T12:37:27.976419+0000","last_became_active":"2024-04-28T12:37:21.799470+0000","last_became_peered":"2024-04-28T12:37:21.799470+0000","last_unstale":"2024-04-28T12:37:27.976419+0000","last_undegraded":"2024-04-28T12:37:27.976419+0000","last_fullsized":"2024-04-28T12:37:27.976419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:49.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977214+0000","last_change":"2024-04-28T12:37:26.874939+0000","last_active":"2024-04-28T12:37:27.977214+0000","last_peered":"2024-04-28T12:37:27.977214+0000","last_clean":"2024-04-28T12:37:27.977214+0000","last_became_active":"2024-04-28T12:37:26.874472+0000","last_became_peered":"2024-04-28T12:37:26.874472+0000","last_unstale":"2024-04-28T12:37:27.977214+0000","last_undegraded":"2024-04-28T12:37:27.977214+0000","last_fullsized":"2024-04-28T12:37:27.977214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981950+0000","last_change":"2024-04-28T12:37:22.814584+0000","last_active":"2024-04-28T12:37:27.981950+0000","last_peered":"2024-04-28T12:37:27.981950+0000","last_clean":"2024-04-28T12:37:27.981950+0000","last_became_active":"2024-04-28T12:37:22.812752+0000","last_became_peered":"2024-04-28T12:37:22.812752+0000","last_unstale":"2024-04-28T12:37:27.981950+0000","last_undegraded":"2024-04-28T12:37:27.981950+0000","last_fullsized":"2024-04-28T12:37:27.981950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:48:32.880830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840416+0000","last_change":"2024-04-28T12:37:21.801322+0000","last_active":"2024-04-28T12:37:27.840416+0000","last_peered":"2024-04-28T12:37:27.840416+0000","last_clean":"2024-04-28T12:37:27.840416+0000","last_became_active":"2024-04-28T12:37:21.800437+0000","last_became_peered":"2024-04-28T12:37:21.800437+0000","last_unstale":"2024-04-28T12:37:27.840416+0000","last_undegraded":"2024-04-28T12:37:27.840416+0000","last_fullsized":"2024-04-28T12:37:27.840416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:33.616030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978163+0000","last_change":"2024-04-28T12:37:26.885854+0000","last_active":"2024-04-28T12:37:27.978163+0000","last_peered":"2024-04-28T12:37:27.978163+0000","last_clean":"2024-04-28T12:37:27.978163+0000","last_became_active":"2024-04-28T12:37:26.884298+0000","last_became_peered":"2024-04-28T12:37:26.884298+0000","last_unstale":"2024-04-28T12:37:27.978163+0000","last_undegraded":"2024-04-28T12:37:27.978163+0000","last_fullsized":"2024-04-28T12:37:27.978163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:19:01.590267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982107+0000","last_change":"2024-04-28T12:37:22.808698+0000","last_active":"2024-04-28T12:37:27.982107+0000","last_peered":"2024-04-28T12:37:27.982107+0000","last_clean":"2024-04-28T12:37:27.982107+0000","last_became_active":"2024-04-28T12:37:22.808363+0000","last_became_peered":"2024-04-28T12:37:22.808363+0000","last_unstale":"2024-04-28T12:37:27.982107+0000","last_undegraded":"2024-04-28T12:37:27.982107+0000","last_fullsized":"2024-04-28T12:37:27.982107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:47.618755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.846296+0000","last_change":"2024-04-28T12:37:21.794490+0000","last_active":"2024-04-28T12:37:27.846296+0000","last_peered":"2024-04-28T12:37:27.846296+0000","last_clean":"2024-04-28T12:37:27.846296+0000","last_became_active":"2024-04-28T12:37:21.794115+0000","last_became_peered":"2024-04-28T12:37:21.794115+0000","last_unstale":"2024-04-28T12:37:27.846296+0000","last_undegraded":"2024-04-28T12:37:27.846296+0000","last_fullsized":"2024-04-28T12:37:27.846296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:20:04.188478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844144+0000","last_change":"2024-04-28T12:37:26.867905+0000","last_active":"2024-04-28T12:37:27.844144+0000","last_peered":"2024-04-28T12:37:27.844144+0000","last_clean":"2024-04-28T12:37:27.844144+0000","last_became_active":"2024-04-28T12:37:26.867317+0000","last_became_peered":"2024-04-28T12:37:26.867317+0000","last_unstale":"2024-04-28T12:37:27.844144+0000","last_undegraded":"2024-04-28T12:37:27.844144+0000","last_fullsized":"2024-04-28T12:37:27.844144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:49.659072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980575+0000","last_change":"2024-04-28T12:37:22.802406+0000","last_active":"2024-04-28T12:37:27.980575+0000","last_peered":"2024-04-28T12:37:27.980575+0000","last_clean":"2024-04-28T12:37:27.980575+0000","last_became_active":"2024-04-28T12:37:22.802094+0000","last_became_peered":"2024-04-28T12:37:22.802094+0000","last_unstale":"2024-04-28T12:37:27.980575+0000","last_undegraded":"2024-04-28T12:37:27.980575+0000","last_fullsized":"2024-04-28T12:37:27.980575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:16.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.838661+0000","last_change":"2024-04-28T12:37:21.788140+0000","last_active":"2024-04-28T12:37:27.838661+0000","last_peered":"2024-04-28T12:37:27.838661+0000","last_clean":"2024-04-28T12:37:27.838661+0000","last_became_active":"2024-04-28T12:37:21.787727+0000","last_became_peered":"2024-04-28T12:37:21.787727+0000","last_unstale":"2024-04-28T12:37:27.838661+0000","last_undegraded":"2024-04-28T12:37:27.838661+0000","last_fullsized":"2024-04-28T12:37:27.838661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:29:20.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978548+0000","last_change":"2024-04-28T12:37:26.868358+0000","last_active":"2024-04-28T12:37:27.978548+0000","last_peered":"2024-04-28T12:37:27.978548+0000","last_clean":"2024-04-28T12:37:27.978548+0000","last_became_active":"2024-04-28T12:37:26.867972+0000","last_became_peered":"2024-04-28T12:37:26.867972+0000","last_unstale":"2024-04-28T12:37:27.978548+0000","last_undegraded":"2024-04-28T12:37:27.978548+0000","last_fullsized":"2024-04-28T12:37:27.978548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:28.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842120+0000","last_change":"2024-04-28T12:37:22.786192+0000","last_active":"2024-04-28T12:37:27.842120+0000","last_peered":"2024-04-28T12:37:27.842120+0000","last_clean":"2024-04-28T12:37:27.842120+0000","last_became_active":"2024-04-28T12:37:22.785730+0000","last_became_peered":"2024-04-28T12:37:22.785730+0000","last_unstale":"2024-04-28T12:37:27.842120+0000","last_undegraded":"2024-04-28T12:37:27.842120+0000","last_fullsized":"2024-04-28T12:37:27.842120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:42.893320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842909+0000","last_change":"2024-04-28T12:37:21.800159+0000","last_active":"2024-04-28T12:37:27.842909+0000","last_peered":"2024-04-28T12:37:27.842909+0000","last_clean":"2024-04-28T12:37:27.842909+0000","last_became_active":"2024-04-28T12:37:21.799817+0000","last_became_peered":"2024-04-28T12:37:21.799817+0000","last_unstale":"2024-04-28T12:37:27.842909+0000","last_undegraded":"2024-04-28T12:37:27.842909+0000","last_fullsized":"2024-04-28T12:37:27.842909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978507+0000","last_change":"2024-04-28T12:37:26.883264+0000","last_active":"2024-04-28T12:37:27.978507+0000","last_peered":"2024-04-28T12:37:27.978507+0000","last_clean":"2024-04-28T12:37:27.978507+0000","last_became_active":"2024-04-28T12:37:26.882787+0000","last_became_peered":"2024-04-28T12:37:26.882787+0000","last_unstale":"2024-04-28T12:37:27.978507+0000","last_undegraded":"2024-04-28T12:37:27.978507+0000","last_fullsized":"2024-04-28T12:37:27.978507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:11:24.219747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981395+0000","last_change":"2024-04-28T12:37:22.811865+0000","last_active":"2024-04-28T12:37:27.981395+0000","last_peered":"2024-04-28T12:37:27.981395+0000","last_clean":"2024-04-28T12:37:27.981395+0000","last_became_active":"2024-04-28T12:37:22.809479+0000","last_became_peered":"2024-04-28T12:37:22.809479+0000","last_unstale":"2024-04-28T12:37:27.981395+0000","last_undegraded":"2024-04-28T12:37:27.981395+0000","last_fullsized":"2024-04-28T12:37:27.981395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:39.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978873+0000","last_change":"2024-04-28T12:37:21.805620+0000","last_active":"2024-04-28T12:37:27.978873+0000","last_peered":"2024-04-28T12:37:27.978873+0000","last_clean":"2024-04-28T12:37:27.978873+0000","last_became_active":"2024-04-28T12:37:21.805397+0000","last_became_peered":"2024-04-28T12:37:21.805397+0000","last_unstale":"2024-04-28T12:37:27.978873+0000","last_undegraded":"2024-04-28T12:37:27.978873+0000","last_fullsized":"2024-04-28T12:37:27.978873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978474+0000","last_change":"2024-04-28T12:37:26.880544+0000","last_active":"2024-04-28T12:37:27.978474+0000","last_peered":"2024-04-28T12:37:27.978474+0000","last_clean":"2024-04-28T12:37:27.978474+0000","last_became_active":"2024-04-28T12:37:26.878960+0000","last_became_peered":"2024-04-28T12:37:26.878960+0000","last_unstale":"2024-04-28T12:37:27.978474+0000","last_undegraded":"2024-04-28T12:37:27.978474+0000","last_fullsized":"2024-04-28T12:37:27.978474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:28:21.376871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977145+0000","last_change":"2024-04-28T12:37:22.795607+0000","last_active":"2024-04-28T12:37:27.977145+0000","last_peered":"2024-04-28T12:37:27.977145+0000","last_clean":"2024-04-28T12:37:27.977145+0000","last_became_active":"2024-04-28T12:37:22.795262+0000","last_became_peered":"2024-04-28T12:37:22.795262+0000","last_unstale":"2024-04-28T12:37:27.977145+0000","last_undegraded":"2024-04-28T12:37:27.977145+0000","last_fullsized":"2024-04-28T12:37:27.977145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:42.893320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977438+0000","last_change":"2024-04-28T12:37:21.798231+0000","last_active":"2024-04-28T12:37:27.977438+0000","last_peered":"2024-04-28T12:37:27.977438+0000","last_clean":"2024-04-28T12:37:27.977438+0000","last_became_active":"2024-04-28T12:37:21.797978+0000","last_became_peered":"2024-04-28T12:37:21.797978+0000","last_unstale":"2024-04-28T12:37:27.977438+0000","last_undegraded":"2024-04-28T12:37:27.977438+0000","last_fullsized":"2024-04-28T12:37:27.977438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:20:04.188478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980535+0000","last_change":"2024-04-28T12:37:26.881273+0000","last_active":"2024-04-28T12:37:27.980535+0000","last_peered":"2024-04-28T12:37:27.980535+0000","last_clean":"2024-04-28T12:37:27.980535+0000","last_became_active":"2024-04-28T12:37:26.880845+0000","last_became_peered":"2024-04-28T12:37:26.880845+0000","last_unstale":"2024-04-28T12:37:27.980535+0000","last_undegraded":"2024-04-28T12:37:27.980535+0000","last_fullsized":"2024-04-28T12:37:27.980535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:09.634939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981232+0000","last_change":"2024-04-28T12:37:22.811901+0000","last_active":"2024-04-28T12:37:27.981232+0000","last_peered":"2024-04-28T12:37:27.981232+0000","last_clean":"2024-04-28T12:37:27.981232+0000","last_became_active":"2024-04-28T12:37:22.807998+0000","last_became_peered":"2024-04-28T12:37:22.807998+0000","last_unstale":"2024-04-28T12:37:27.981232+0000","last_undegraded":"2024-04-28T12:37:27.981232+0000","last_fullsized":"2024-04-28T12:37:27.981232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:29:25.559764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:41.723137+0000","last_change":"2024-04-28T12:37:21.802895+0000","last_active":"2024-04-28T12:37:41.723137+0000","last_peered":"2024-04-28T12:37:41.723137+0000","last_clean":"2024-04-28T12:37:41.723137+0000","last_became_active":"2024-04-28T12:37:21.801912+0000","last_became_peered":"2024-04-28T12:37:21.801912+0000","last_unstale":"2024-04-28T12:37:41.723137+0000","last_undegraded":"2024-04-28T12:37:41.723137+0000","last_fullsized":"2024-04-28T12:37:41.723137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:39.334819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977722+0000","last_change":"2024-04-28T12:37:26.873444+0000","last_active":"2024-04-28T12:37:27.977722+0000","last_peered":"2024-04-28T12:37:27.977722+0000","last_clean":"2024-04-28T12:37:27.977722+0000","last_became_active":"2024-04-28T12:37:26.872331+0000","last_became_peered":"2024-04-28T12:37:26.872331+0000","last_unstale":"2024-04-28T12:37:27.977722+0000","last_undegraded":"2024-04-28T12:37:27.977722+0000","last_fullsized":"2024-04-28T12:37:27.977722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:51.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845884+0000","last_change":"2024-04-28T12:37:22.786473+0000","last_active":"2024-04-28T12:37:27.845884+0000","last_peered":"2024-04-28T12:37:27.845884+0000","last_clean":"2024-04-28T12:37:27.845884+0000","last_became_active":"2024-04-28T12:37:22.785266+0000","last_became_peered":"2024-04-28T12:37:22.785266+0000","last_unstale":"2024-04-28T12:37:27.845884+0000","last_undegraded":"2024-04-28T12:37:27.845884+0000","last_fullsized":"2024-04-28T12:37:27.845884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:50.704472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976275+0000","last_change":"2024-04-28T12:37:21.810514+0000","last_active":"2024-04-28T12:37:27.976275+0000","last_peered":"2024-04-28T12:37:27.976275+0000","last_clean":"2024-04-28T12:37:27.976275+0000","last_became_active":"2024-04-28T12:37:21.810221+0000","last_became_peered":"2024-04-28T12:37:21.810221+0000","last_unstale":"2024-04-28T12:37:27.976275+0000","last_undegraded":"2024-04-28T12:37:27.976275+0000","last_fullsized":"2024-04-28T12:37:27.976275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845261+0000","last_change":"2024-04-28T12:37:26.871384+0000","last_active":"2024-04-28T12:37:27.845261+0000","last_peered":"2024-04-28T12:37:27.845261+0000","last_clean":"2024-04-28T12:37:27.845261+0000","last_became_active":"2024-04-28T12:37:26.868701+0000","last_became_peered":"2024-04-28T12:37:26.868701+0000","last_unstale":"2024-04-28T12:37:27.845261+0000","last_undegraded":"2024-04-28T12:37:27.845261+0000","last_fullsized":"2024-04-28T12:37:27.845261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:44.015884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979855+0000","last_change":"2024-04-28T12:37:22.821706+0000","last_active":"2024-04-28T12:37:27.979855+0000","last_peered":"2024-04-28T12:37:27.979855+0000","last_clean":"2024-04-28T12:37:27.979855+0000","last_became_active":"2024-04-28T12:37:22.821338+0000","last_became_peered":"2024-04-28T12:37:22.821338+0000","last_unstale":"2024-04-28T12:37:27.979855+0000","last_undegraded":"2024-04-28T12:37:27.979855+0000","last_fullsized":"2024-04-28T12:37:27.979855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980054+0000","last_change":"2024-04-28T12:37:21.803535+0000","last_active":"2024-04-28T12:37:27.980054+0000","last_peered":"2024-04-28T12:37:27.980054+0000","last_clean":"2024-04-28T12:37:27.980054+0000","last_became_active":"2024-04-28T12:37:21.801895+0000","last_became_peered":"2024-04-28T12:37:21.801895+0000","last_unstale":"2024-04-28T12:37:27.980054+0000","last_undegraded":"2024-04-28T12:37:27.980054+0000","last_fullsized":"2024-04-28T12:37:27.980054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:32.253270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976957+0000","last_change":"2024-04-28T12:37:26.861154+0000","last_active":"2024-04-28T12:37:27.976957+0000","last_peered":"2024-04-28T12:37:27.976957+0000","last_clean":"2024-04-28T12:37:27.976957+0000","last_became_active":"2024-04-28T12:37:26.860732+0000","last_became_peered":"2024-04-28T12:37:26.860732+0000","last_unstale":"2024-04-28T12:37:27.976957+0000","last_undegraded":"2024-04-28T12:37:27.976957+0000","last_fullsized":"2024-04-28T12:37:27.976957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:48:40.360635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979341+0000","last_change":"2024-04-28T12:37:22.817129+0000","last_active":"2024-04-28T12:37:27.979341+0000","last_peered":"2024-04-28T12:37:27.979341+0000","last_clean":"2024-04-28T12:37:27.979341+0000","last_became_active":"2024-04-28T12:37:22.816811+0000","last_became_peered":"2024-04-28T12:37:22.816811+0000","last_unstale":"2024-04-28T12:37:27.979341+0000","last_undegraded":"2024-04-28T12:37:27.979341+0000","last_fullsized":"2024-04-28T12:37:27.979341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:29:25.559764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843391+0000","last_change":"2024-04-28T12:37:21.805873+0000","last_active":"2024-04-28T12:37:27.843391+0000","last_peered":"2024-04-28T12:37:27.843391+0000","last_clean":"2024-04-28T12:37:27.843391+0000","last_became_active":"2024-04-28T12:37:21.805549+0000","last_became_peered":"2024-04-28T12:37:21.805549+0000","last_unstale":"2024-04-28T12:37:27.843391+0000","last_undegraded":"2024-04-28T12:37:27.843391+0000","last_fullsized":"2024-04-28T12:37:27.843391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976927+0000","last_change":"2024-04-28T12:37:26.883208+0000","last_active":"2024-04-28T12:37:27.976927+0000","last_peered":"2024-04-28T12:37:27.976927+0000","last_clean":"2024-04-28T12:37:27.976927+0000","last_became_active":"2024-04-28T12:37:26.882732+0000","last_became_peered":"2024-04-28T12:37:26.882732+0000","last_unstale":"2024-04-28T12:37:27.976927+0000","last_undegraded":"2024-04-28T12:37:27.976927+0000","last_fullsized":"2024-04-28T12:37:27.976927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:37:09.856024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978306+0000","last_change":"2024-04-28T12:37:26.888160+0000","last_active":"2024-04-28T12:37:27.978306+0000","last_peered":"2024-04-28T12:37:27.978306+0000","last_clean":"2024-04-28T12:37:27.978306+0000","last_became_active":"2024-04-28T12:37:26.886631+0000","last_became_peered":"2024-04-28T12:37:26.886631+0000","last_unstale":"2024-04-28T12:37:27.978306+0000","last_undegraded":"2024-04-28T12:37:27.978306+0000","last_fullsized":"2024-04-28T12:37:27.978306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:51.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.975748+0000","last_change":"2024-04-28T12:37:22.794465+0000","last_active":"2024-04-28T12:37:27.975748+0000","last_peered":"2024-04-28T12:37:27.975748+0000","last_clean":"2024-04-28T12:37:27.975748+0000","last_became_active":"2024-04-28T12:37:22.794138+0000","last_became_peered":"2024-04-28T12:37:22.794138+0000","last_unstale":"2024-04-28T12:37:27.975748+0000","last_undegraded":"2024-04-28T12:37:27.975748+0000","last_fullsized":"2024-04-28T12:37:27.975748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978354+0000","last_change":"2024-04-28T12:37:21.805697+0000","last_active":"2024-04-28T12:37:27.978354+0000","last_peered":"2024-04-28T12:37:27.978354+0000","last_clean":"2024-04-28T12:37:27.978354+0000","last_became_active":"2024-04-28T12:37:21.805449+0000","last_became_peered":"2024-04-28T12:37:21.805449+0000","last_unstale":"2024-04-28T12:37:27.978354+0000","last_undegraded":"2024-04-28T12:37:27.978354+0000","last_fullsized":"2024-04-28T12:37:27.978354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843785+0000","last_change":"2024-04-28T12:37:26.869931+0000","last_active":"2024-04-28T12:37:27.843785+0000","last_peered":"2024-04-28T12:37:27.843785+0000","last_clean":"2024-04-28T12:37:27.843785+0000","last_became_active":"2024-04-28T12:37:26.869441+0000","last_became_peered":"2024-04-28T12:37:26.869441+0000","last_unstale":"2024-04-28T12:37:27.843785+0000","last_undegraded":"2024-04-28T12:37:27.843785+0000","last_fullsized":"2024-04-28T12:37:27.843785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977035+0000","last_change":"2024-04-28T12:37:22.792490+0000","last_active":"2024-04-28T12:37:27.977035+0000","last_peered":"2024-04-28T12:37:27.977035+0000","last_clean":"2024-04-28T12:37:27.977035+0000","last_became_active":"2024-04-28T12:37:22.790799+0000","last_became_peered":"2024-04-28T12:37:22.790799+0000","last_unstale":"2024-04-28T12:37:27.977035+0000","last_undegraded":"2024-04-28T12:37:27.977035+0000","last_fullsized":"2024-04-28T12:37:27.977035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977471+0000","last_change":"2024-04-28T12:37:21.799595+0000","last_active":"2024-04-28T12:37:27.977471+0000","last_peered":"2024-04-28T12:37:27.977471+0000","last_clean":"2024-04-28T12:37:27.977471+0000","last_became_active":"2024-04-28T12:37:21.796500+0000","last_became_peered":"2024-04-28T12:37:21.796500+0000","last_unstale":"2024-04-28T12:37:27.977471+0000","last_undegraded":"2024-04-28T12:37:27.977471+0000","last_fullsized":"2024-04-28T12:37:27.977471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:33.616030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843252+0000","last_change":"2024-04-28T12:37:21.799592+0000","last_active":"2024-04-28T12:37:27.843252+0000","last_peered":"2024-04-28T12:37:27.843252+0000","last_clean":"2024-04-28T12:37:27.843252+0000","last_became_active":"2024-04-28T12:37:21.799085+0000","last_became_peered":"2024-04-28T12:37:21.799085+0000","last_unstale":"2024-04-28T12:37:27.843252+0000","last_undegraded":"2024-04-28T12:37:27.843252+0000","last_fullsized":"2024-04-28T12:37:27.843252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:53.650061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977307+0000","last_change":"2024-04-28T12:37:22.781856+0000","last_active":"2024-04-28T12:37:27.977307+0000","last_peered":"2024-04-28T12:37:27.977307+0000","last_clean":"2024-04-28T12:37:27.977307+0000","last_became_active":"2024-04-28T12:37:22.780347+0000","last_became_peered":"2024-04-28T12:37:22.780347+0000","last_unstale":"2024-04-28T12:37:27.977307+0000","last_undegraded":"2024-04-28T12:37:27.977307+0000","last_fullsized":"2024-04-28T12:37:27.977307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840176+0000","last_change":"2024-04-28T12:37:26.867761+0000","last_active":"2024-04-28T12:37:27.840176+0000","last_peered":"2024-04-28T12:37:27.840176+0000","last_clean":"2024-04-28T12:37:27.840176+0000","last_became_active":"2024-04-28T12:37:26.867277+0000","last_became_peered":"2024-04-28T12:37:26.867277+0000","last_unstale":"2024-04-28T12:37:27.840176+0000","last_undegraded":"2024-04-28T12:37:27.840176+0000","last_fullsized":"2024-04-28T12:37:27.840176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:12:40.786592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977843+0000","last_change":"2024-04-28T12:37:21.812666+0000","last_active":"2024-04-28T12:37:27.977843+0000","last_peered":"2024-04-28T12:37:27.977843+0000","last_clean":"2024-04-28T12:37:27.977843+0000","last_became_active":"2024-04-28T12:37:21.812384+0000","last_became_peered":"2024-04-28T12:37:21.812384+0000","last_unstale":"2024-04-28T12:37:27.977843+0000","last_undegraded":"2024-04-28T12:37:27.977843+0000","last_fullsized":"2024-04-28T12:37:27.977843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:29:20.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844555+0000","last_change":"2024-04-28T12:37:22.794127+0000","last_active":"2024-04-28T12:37:27.844555+0000","last_peered":"2024-04-28T12:37:27.844555+0000","last_clean":"2024-04-28T12:37:27.844555+0000","last_became_active":"2024-04-28T12:37:22.793770+0000","last_became_peered":"2024-04-28T12:37:22.793770+0000","last_unstale":"2024-04-28T12:37:27.844555+0000","last_undegraded":"2024-04-28T12:37:27.844555+0000","last_fullsized":"2024-04-28T12:37:27.844555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:00:05.608247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977959+0000","last_change":"2024-04-28T12:37:26.863918+0000","last_active":"2024-04-28T12:37:27.977959+0000","last_peered":"2024-04-28T12:37:27.977959+0000","last_clean":"2024-04-28T12:37:27.977959+0000","last_became_active":"2024-04-28T12:37:26.863558+0000","last_became_peered":"2024-04-28T12:37:26.863558+0000","last_unstale":"2024-04-28T12:37:27.977959+0000","last_undegraded":"2024-04-28T12:37:27.977959+0000","last_fullsized":"2024-04-28T12:37:27.977959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:49:38.590277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.841795+0000","last_change":"2024-04-28T12:37:22.786731+0000","last_active":"2024-04-28T12:37:27.841795+0000","last_peered":"2024-04-28T12:37:27.841795+0000","last_clean":"2024-04-28T12:37:27.841795+0000","last_became_active":"2024-04-28T12:37:22.786283+0000","last_became_peered":"2024-04-28T12:37:22.786283+0000","last_unstale":"2024-04-28T12:37:27.841795+0000","last_undegraded":"2024-04-28T12:37:27.841795+0000","last_fullsized":"2024-04-28T12:37:27.841795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.838968+0000","last_change":"2024-04-28T12:37:22.793953+0000","last_active":"2024-04-28T12:37:27.838968+0000","last_peered":"2024-04-28T12:37:27.838968+0000","last_clean":"2024-04-28T12:37:27.838968+0000","last_became_active":"2024-04-28T12:37:22.793683+0000","last_became_peered":"2024-04-28T12:37:22.793683+0000","last_unstale":"2024-04-28T12:37:27.838968+0000","last_undegraded":"2024-04-28T12:37:27.838968+0000","last_fullsized":"2024-04-28T12:37:27.838968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:33.269962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978735+0000","last_change":"2024-04-28T12:37:22.791966+0000","last_active":"2024-04-28T12:37:27.978735+0000","last_peered":"2024-04-28T12:37:27.978735+0000","last_clean":"2024-04-28T12:37:27.978735+0000","last_became_active":"2024-04-28T12:37:22.791432+0000","last_became_peered":"2024-04-28T12:37:22.791432+0000","last_unstale":"2024-04-28T12:37:27.978735+0000","last_undegraded":"2024-04-28T12:37:27.978735+0000","last_fullsized":"2024-04-28T12:37:27.978735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:47.693882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977566+0000","last_change":"2024-04-28T12:37:22.786169+0000","last_active":"2024-04-28T12:37:27.977566+0000","last_peered":"2024-04-28T12:37:27.977566+0000","last_clean":"2024-04-28T12:37:27.977566+0000","last_became_active":"2024-04-28T12:37:22.785899+0000","last_became_peered":"2024-04-28T12:37:22.785899+0000","last_unstale":"2024-04-28T12:37:27.977566+0000","last_undegraded":"2024-04-28T12:37:27.977566+0000","last_fullsized":"2024-04-28T12:37:27.977566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:06:12.328002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979102+0000","last_change":"2024-04-28T12:37:22.812114+0000","last_active":"2024-04-28T12:37:27.979102+0000","last_peered":"2024-04-28T12:37:27.979102+0000","last_clean":"2024-04-28T12:37:27.979102+0000","last_became_active":"2024-04-28T12:37:22.808394+0000","last_became_peered":"2024-04-28T12:37:22.808394+0000","last_unstale":"2024-04-28T12:37:27.979102+0000","last_undegraded":"2024-04-28T12:37:27.979102+0000","last_fullsized":"2024-04-28T12:37:27.979102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976044+0000","last_change":"2024-04-28T12:37:22.803864+0000","last_active":"2024-04-28T12:37:27.976044+0000","last_peered":"2024-04-28T12:37:27.976044+0000","last_clean":"2024-04-28T12:37:27.976044+0000","last_became_active":"2024-04-28T12:37:22.803568+0000","last_became_peered":"2024-04-28T12:37:22.803568+0000","last_unstale":"2024-04-28T12:37:27.976044+0000","last_undegraded":"2024-04-28T12:37:27.976044+0000","last_fullsized":"2024-04-28T12:37:27.976044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844461+0000","last_change":"2024-04-28T12:37:22.788814+0000","last_active":"2024-04-28T12:37:27.844461+0000","last_peered":"2024-04-28T12:37:27.844461+0000","last_clean":"2024-04-28T12:37:27.844461+0000","last_became_active":"2024-04-28T12:37:22.786716+0000","last_became_peered":"2024-04-28T12:37:22.786716+0000","last_unstale":"2024-04-28T12:37:27.844461+0000","last_undegraded":"2024-04-28T12:37:27.844461+0000","last_fullsized":"2024-04-28T12:37:27.844461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978543+0000","last_change":"2024-04-28T12:37:22.812645+0000","last_active":"2024-04-28T12:37:27.978543+0000","last_peered":"2024-04-28T12:37:27.978543+0000","last_clean":"2024-04-28T12:37:27.978543+0000","last_became_active":"2024-04-28T12:37:22.811034+0000","last_became_peered":"2024-04-28T12:37:22.811034+0000","last_unstale":"2024-04-28T12:37:27.978543+0000","last_undegraded":"2024-04-28T12:37:27.978543+0000","last_fullsized":"2024-04-28T12:37:27.978543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978281+0000","last_change":"2024-04-28T12:37:22.804060+0000","last_active":"2024-04-28T12:37:27.978281+0000","last_peered":"2024-04-28T12:37:27.978281+0000","last_clean":"2024-04-28T12:37:27.978281+0000","last_became_active":"2024-04-28T12:37:22.803757+0000","last_became_peered":"2024-04-28T12:37:22.803757+0000","last_unstale":"2024-04-28T12:37:27.978281+0000","last_undegraded":"2024-04-28T12:37:27.978281+0000","last_fullsized":"2024-04-28T12:37:27.978281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977634+0000","last_change":"2024-04-28T12:37:22.795301+0000","last_active":"2024-04-28T12:37:27.977634+0000","last_peered":"2024-04-28T12:37:27.977634+0000","last_clean":"2024-04-28T12:37:27.977634+0000","last_became_active":"2024-04-28T12:37:22.794917+0000","last_became_peered":"2024-04-28T12:37:22.794917+0000","last_unstale":"2024-04-28T12:37:27.977634+0000","last_undegraded":"2024-04-28T12:37:27.977634+0000","last_fullsized":"2024-04-28T12:37:27.977634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:45.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978597+0000","last_change":"2024-04-28T12:37:22.805760+0000","last_active":"2024-04-28T12:37:27.978597+0000","last_peered":"2024-04-28T12:37:27.978597+0000","last_clean":"2024-04-28T12:37:27.978597+0000","last_became_active":"2024-04-28T12:37:22.805238+0000","last_became_peered":"2024-04-28T12:37:22.805238+0000","last_unstale":"2024-04-28T12:37:27.978597+0000","last_undegraded":"2024-04-28T12:37:27.978597+0000","last_fullsized":"2024-04-28T12:37:27.978597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:16.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976930+0000","last_change":"2024-04-28T12:37:22.793207+0000","last_active":"2024-04-28T12:37:27.976930+0000","last_peered":"2024-04-28T12:37:27.976930+0000","last_clean":"2024-04-28T12:37:27.976930+0000","last_became_active":"2024-04-28T12:37:22.792816+0000","last_became_peered":"2024-04-28T12:37:22.792816+0000","last_unstale":"2024-04-28T12:37:27.976930+0000","last_undegraded":"2024-04-28T12:37:27.976930+0000","last_fullsized":"2024-04-28T12:37:27.976930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:55:36.416184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842925+0000","last_change":"2024-04-28T12:37:22.780772+0000","last_active":"2024-04-28T12:37:27.842925+0000","last_peered":"2024-04-28T12:37:27.842925+0000","last_clean":"2024-04-28T12:37:27.842925+0000","last_became_active":"2024-04-28T12:37:22.780190+0000","last_became_peered":"2024-04-28T12:37:22.780190+0000","last_unstale":"2024-04-28T12:37:27.842925+0000","last_undegraded":"2024-04-28T12:37:27.842925+0000","last_fullsized":"2024-04-28T12:37:27.842925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982761+0000","last_change":"2024-04-28T12:37:22.799792+0000","last_active":"2024-04-28T12:37:27.982761+0000","last_peered":"2024-04-28T12:37:27.982761+0000","last_clean":"2024-04-28T12:37:27.982761+0000","last_became_active":"2024-04-28T12:37:22.799472+0000","last_became_peered":"2024-04-28T12:37:22.799472+0000","last_unstale":"2024-04-28T12:37:27.982761+0000","last_undegraded":"2024-04-28T12:37:27.982761+0000","last_fullsized":"2024-04-28T12:37:27.982761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:47.693882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840300+0000","last_change":"2024-04-28T12:37:22.793748+0000","last_active":"2024-04-28T12:37:27.840300+0000","last_peered":"2024-04-28T12:37:27.840300+0000","last_clean":"2024-04-28T12:37:27.840300+0000","last_became_active":"2024-04-28T12:37:22.793461+0000","last_became_peered":"2024-04-28T12:37:22.793461+0000","last_unstale":"2024-04-28T12:37:27.840300+0000","last_undegraded":"2024-04-28T12:37:27.840300+0000","last_fullsized":"2024-04-28T12:37:27.840300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:00:05.608247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980132+0000","last_change":"2024-04-28T12:37:22.816550+0000","last_active":"2024-04-28T12:37:27.980132+0000","last_peered":"2024-04-28T12:37:27.980132+0000","last_clean":"2024-04-28T12:37:27.980132+0000","last_became_active":"2024-04-28T12:37:22.811895+0000","last_became_peered":"2024-04-28T12:37:22.811895+0000","last_unstale":"2024-04-28T12:37:27.980132+0000","last_undegraded":"2024-04-28T12:37:27.980132+0000","last_fullsized":"2024-04-28T12:37:27.980132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:24.827452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977952+0000","last_change":"2024-04-28T12:37:22.803909+0000","last_active":"2024-04-28T12:37:27.977952+0000","last_peered":"2024-04-28T12:37:27.977952+0000","last_clean":"2024-04-28T12:37:27.977952+0000","last_became_active":"2024-04-28T12:37:22.803577+0000","last_became_peered":"2024-04-28T12:37:22.803577+0000","last_unstale":"2024-04-28T12:37:27.977952+0000","last_undegraded":"2024-04-28T12:37:27.977952+0000","last_fullsized":"2024-04-28T12:37:27.977952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:59.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.975883+0000","last_change":"2024-04-28T12:37:22.808602+0000","last_active":"2024-04-28T12:37:27.975883+0000","last_peered":"2024-04-28T12:37:27.975883+0000","last_clean":"2024-04-28T12:37:27.975883+0000","last_became_active":"2024-04-28T12:37:22.808300+0000","last_became_peered":"2024-04-28T12:37:22.808300+0000","last_unstale":"2024-04-28T12:37:27.975883+0000","last_undegraded":"2024-04-28T12:37:27.975883+0000","last_fullsized":"2024-04-28T12:37:27.975883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977024+0000","last_change":"2024-04-28T12:37:22.792556+0000","last_active":"2024-04-28T12:37:27.977024+0000","last_peered":"2024-04-28T12:37:27.977024+0000","last_clean":"2024-04-28T12:37:27.977024+0000","last_became_active":"2024-04-28T12:37:22.789568+0000","last_became_peered":"2024-04-28T12:37:22.789568+0000","last_unstale":"2024-04-28T12:37:27.977024+0000","last_undegraded":"2024-04-28T12:37:27.977024+0000","last_fullsized":"2024-04-28T12:37:27.977024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:12:36.729037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977547+0000","last_change":"2024-04-28T12:37:22.798420+0000","last_active":"2024-04-28T12:37:27.977547+0000","last_peered":"2024-04-28T12:37:27.977547+0000","last_clean":"2024-04-28T12:37:27.977547+0000","last_became_active":"2024-04-28T12:37:22.798125+0000","last_became_peered":"2024-04-28T12:37:22.798125+0000","last_unstale":"2024-04-28T12:37:27.977547+0000","last_undegraded":"2024-04-28T12:37:27.977547+0000","last_fullsized":"2024-04-28T12:37:27.977547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:42.893320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845776+0000","last_change":"2024-04-28T12:37:22.799783+0000","last_active":"2024-04-28T12:37:27.845776+0000","last_peered":"2024-04-28T12:37:27.845776+0000","last_clean":"2024-04-28T12:37:27.845776+0000","last_became_active":"2024-04-28T12:37:22.796468+0000","last_became_peered":"2024-04-28T12:37:22.796468+0000","last_unstale":"2024-04-28T12:37:27.845776+0000","last_undegraded":"2024-04-28T12:37:27.845776+0000","last_fullsized":"2024-04-28T12:37:27.845776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:00.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844178+0000","last_change":"2024-04-28T12:37:22.806029+0000","last_active":"2024-04-28T12:37:27.844178+0000","last_peered":"2024-04-28T12:37:27.844178+0000","last_clean":"2024-04-28T12:37:27.844178+0000","last_became_active":"2024-04-28T12:37:22.802600+0000","last_became_peered":"2024-04-28T12:37:22.802600+0000","last_unstale":"2024-04-28T12:37:27.844178+0000","last_undegraded":"2024-04-28T12:37:27.844178+0000","last_fullsized":"2024-04-28T12:37:27.844178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844198+0000","last_change":"2024-04-28T12:37:22.821112+0000","last_active":"2024-04-28T12:37:27.844198+0000","last_peered":"2024-04-28T12:37:27.844198+0000","last_clean":"2024-04-28T12:37:27.844198+0000","last_became_active":"2024-04-28T12:37:22.820777+0000","last_became_peered":"2024-04-28T12:37:22.820777+0000","last_unstale":"2024-04-28T12:37:27.844198+0000","last_undegraded":"2024-04-28T12:37:27.844198+0000","last_fullsized":"2024-04-28T12:37:27.844198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981775+0000","last_change":"2024-04-28T12:37:22.825673+0000","last_active":"2024-04-28T12:37:27.981775+0000","last_peered":"2024-04-28T12:37:27.981775+0000","last_clean":"2024-04-28T12:37:27.981775+0000","last_became_active":"2024-04-28T12:37:22.820991+0000","last_became_peered":"2024-04-28T12:37:22.820991+0000","last_unstale":"2024-04-28T12:37:27.981775+0000","last_undegraded":"2024-04-28T12:37:27.981775+0000","last_fullsized":"2024-04-28T12:37:27.981775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:00.502362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976734+0000","last_change":"2024-04-28T12:37:22.814076+0000","last_active":"2024-04-28T12:37:27.976734+0000","last_peered":"2024-04-28T12:37:27.976734+0000","last_clean":"2024-04-28T12:37:27.976734+0000","last_became_active":"2024-04-28T12:37:22.813739+0000","last_became_peered":"2024-04-28T12:37:22.813739+0000","last_unstale":"2024-04-28T12:37:27.976734+0000","last_undegraded":"2024-04-28T12:37:27.976734+0000","last_fullsized":"2024-04-28T12:37:27.976734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980558+0000","last_change":"2024-04-28T12:37:22.799783+0000","last_active":"2024-04-28T12:37:27.980558+0000","last_peered":"2024-04-28T12:37:27.980558+0000","last_clean":"2024-04-28T12:37:27.980558+0000","last_became_active":"2024-04-28T12:37:22.799519+0000","last_became_peered":"2024-04-28T12:37:22.799519+0000","last_unstale":"2024-04-28T12:37:27.980558+0000","last_undegraded":"2024-04-28T12:37:27.980558+0000","last_fullsized":"2024-04-28T12:37:27.980558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:59.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978456+0000","last_change":"2024-04-28T12:37:22.805539+0000","last_active":"2024-04-28T12:37:27.978456+0000","last_peered":"2024-04-28T12:37:27.978456+0000","last_clean":"2024-04-28T12:37:27.978456+0000","last_became_active":"2024-04-28T12:37:22.804834+0000","last_became_peered":"2024-04-28T12:37:22.804834+0000","last_unstale":"2024-04-28T12:37:27.978456+0000","last_undegraded":"2024-04-28T12:37:27.978456+0000","last_fullsized":"2024-04-28T12:37:27.978456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:47.618755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981101+0000","last_change":"2024-04-28T12:37:22.800776+0000","last_active":"2024-04-28T12:37:27.981101+0000","last_peered":"2024-04-28T12:37:27.981101+0000","last_clean":"2024-04-28T12:37:27.981101+0000","last_became_active":"2024-04-28T12:37:22.798852+0000","last_became_peered":"2024-04-28T12:37:22.798852+0000","last_unstale":"2024-04-28T12:37:27.981101+0000","last_undegraded":"2024-04-28T12:37:27.981101+0000","last_fullsized":"2024-04-28T12:37:27.981101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:24.827452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978903+0000","last_change":"2024-04-28T12:37:22.819772+0000","last_active":"2024-04-28T12:37:27.978903+0000","last_peered":"2024-04-28T12:37:27.978903+0000","last_clean":"2024-04-28T12:37:27.978903+0000","last_became_active":"2024-04-28T12:37:22.813375+0000","last_became_peered":"2024-04-28T12:37:22.813375+0000","last_unstale":"2024-04-28T12:37:27.978903+0000","last_undegraded":"2024-04-28T12:37:27.978903+0000","last_fullsized":"2024-04-28T12:37:27.978903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978010+0000","last_change":"2024-04-28T12:37:22.784463+0000","last_active":"2024-04-28T12:37:27.978010+0000","last_peered":"2024-04-28T12:37:27.978010+0000","last_clean":"2024-04-28T12:37:27.978010+0000","last_became_active":"2024-04-28T12:37:22.784183+0000","last_became_peered":"2024-04-28T12:37:22.784183+0000","last_unstale":"2024-04-28T12:37:27.978010+0000","last_undegraded":"2024-04-28T12:37:27.978010+0000","last_fullsized":"2024-04-28T12:37:27.978010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976443+0000","last_change":"2024-04-28T12:37:22.802869+0000","last_active":"2024-04-28T12:37:27.976443+0000","last_peered":"2024-04-28T12:37:27.976443+0000","last_clean":"2024-04-28T12:37:27.976443+0000","last_became_active":"2024-04-28T12:37:22.802598+0000","last_became_peered":"2024-04-28T12:37:22.802598+0000","last_unstale":"2024-04-28T12:37:27.976443+0000","last_undegraded":"2024-04-28T12:37:27.976443+0000","last_fullsized":"2024-04-28T12:37:27.976443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981568+0000","last_change":"2024-04-28T12:37:22.797900+0000","last_active":"2024-04-28T12:37:27.981568+0000","last_peered":"2024-04-28T12:37:27.981568+0000","last_clean":"2024-04-28T12:37:27.981568+0000","last_became_active":"2024-04-28T12:37:22.797533+0000","last_became_peered":"2024-04-28T12:37:22.797533+0000","last_unstale":"2024-04-28T12:37:27.981568+0000","last_undegraded":"2024-04-28T12:37:27.981568+0000","last_fullsized":"2024-04-28T12:37:27.981568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:08:46.974988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:37:52.993 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-28T12:37:52.994 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-28T12:37:52.994 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-28T12:37:52.994 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-28T12:37:52.994 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-28T12:37:53.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.236+0000 7ff399c00640 1 Processor -- start 2024-04-28T12:37:53.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.236+0000 7ff399c00640 1 -- start start 2024-04-28T12:37:53.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff399c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff39415bb90 0x7ff3940737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff399c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3940af710 0x7ff3940afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff399c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff394072db0 con 0x7ff39415f610 2024-04-28T12:37:53.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff399c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff3940cec10 con 0x7ff3940af710 2024-04-28T12:37:53.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff399c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff3940b00b0 con 0x7ff39415bb90 2024-04-28T12:37:53.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff38be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3940af710 0x7ff3940afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff38be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3940af710 0x7ff3940afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45094/0 (socket says 172.21.15.131:45094) 2024-04-28T12:37:53.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff38be00640 1 -- 172.21.15.131:0/1447901558 learned_addr learned my addr 172.21.15.131:0/1447901558 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:53.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff393e00640 1 --2- 172.21.15.131:0/1447901558 >> v2:172.21.15.157:3300/0 conn(0x7ff39415bb90 0x7ff3940737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff38be00640 1 -- 172.21.15.131:0/1447901558 >> v2:172.21.15.157:3300/0 conn(0x7ff39415bb90 msgr2=0x7ff3940737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:53.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff38be00640 1 --2- 172.21.15.131:0/1447901558 >> v2:172.21.15.157:3300/0 conn(0x7ff39415bb90 0x7ff3940737a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff38be00640 1 -- 172.21.15.131:0/1447901558 >> v1:172.21.15.157:6789/0 conn(0x7ff39415f610 legacy=0x7ff39415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:53.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.238+0000 7ff38be00640 1 -- 172.21.15.131:0/1447901558 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3940cee40 con 0x7ff3940af710 2024-04-28T12:37:53.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.239+0000 7ff38be00640 1 --2- 172.21.15.131:0/1447901558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3940af710 0x7ff3940afb70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff39406f820 tx=0x7ff37c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d0399799add31 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:53.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.240+0000 7ff393400640 1 -- 172.21.15.131:0/1447901558 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c037070 con 0x7ff3940af710 2024-04-28T12:37:53.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.240+0000 7ff393400640 1 -- 172.21.15.131:0/1447901558 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff37c032440 con 0x7ff3940af710 2024-04-28T12:37:53.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.240+0000 7ff393400640 1 -- 172.21.15.131:0/1447901558 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c032810 con 0x7ff3940af710 2024-04-28T12:37:53.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.240+0000 7ff399c00640 1 -- 172.21.15.131:0/1447901558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3940af710 msgr2=0x7ff3940afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:53.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.240+0000 7ff399c00640 1 --2- 172.21.15.131:0/1447901558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3940af710 0x7ff3940afb70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff39406f820 tx=0x7ff37c02a230 comp rx=0 tx=0).stop 2024-04-28T12:37:53.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.240+0000 7ff399c00640 1 -- 172.21.15.131:0/1447901558 shutdown_connections 2024-04-28T12:37:53.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.240+0000 7ff399c00640 1 --2- 172.21.15.131:0/1447901558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3940af710 0x7ff3940afb70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.240+0000 7ff399c00640 1 --2- 172.21.15.131:0/1447901558 >> v2:172.21.15.157:3300/0 conn(0x7ff39415bb90 0x7ff3940737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.240+0000 7ff399c00640 1 -- 172.21.15.131:0/1447901558 >> 172.21.15.131:0/1447901558 conn(0x7ff394068090 msgr2=0x7ff394068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:53.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.240+0000 7ff399c00640 1 -- 172.21.15.131:0/1447901558 shutdown_connections 2024-04-28T12:37:53.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.241+0000 7ff399c00640 1 -- 172.21.15.131:0/1447901558 wait complete. 2024-04-28T12:37:53.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.242+0000 7ff399c00640 1 Processor -- start 2024-04-28T12:37:53.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.242+0000 7ff399c00640 1 -- start start 2024-04-28T12:37:53.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.243+0000 7ff399c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3940af710 0x7ff394143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.243+0000 7ff399c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff39415bb90 0x7ff39414a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.243+0000 7ff399c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff39415f610 0x7ff3940ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.243+0000 7ff399c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff3940d0610 con 0x7ff39415bb90 2024-04-28T12:37:53.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.243+0000 7ff399c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff3940725a0 con 0x7ff3940af710 2024-04-28T12:37:53.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.243+0000 7ff399c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff394072ad0 con 0x7ff39415f610 2024-04-28T12:37:53.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.244+0000 7ff38be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff39415bb90 0x7ff39414a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.244+0000 7ff38be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff39415bb90 0x7ff39414a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45102/0 (socket says 172.21.15.131:45102) 2024-04-28T12:37:53.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.244+0000 7ff38be00640 1 -- 172.21.15.131:0/3614703249 learned_addr learned my addr 172.21.15.131:0/3614703249 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:53.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.244+0000 7ff399200640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff39415f610 0x7ff3940ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.244+0000 7ff393e00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3940af710 0x7ff394143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.244+0000 7ff38be00640 1 -- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff39415f610 msgr2=0x7ff3940ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:53.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.244+0000 7ff38be00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff39415f610 0x7ff3940ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.244+0000 7ff38be00640 1 -- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3940af710 msgr2=0x7ff394143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:53.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.244+0000 7ff38be00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3940af710 0x7ff394143070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.244+0000 7ff38be00640 1 -- 172.21.15.131:0/3614703249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39415c0c0 con 0x7ff39415bb90 2024-04-28T12:37:53.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.244+0000 7ff393e00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3940af710 0x7ff394143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:53.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.244+0000 7ff38be00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff39415bb90 0x7ff39414a350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff37c000c00 tx=0x7ff37c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:53.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.245+0000 7ff390c00640 1 -- 172.21.15.131:0/3614703249 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c037040 con 0x7ff39415bb90 2024-04-28T12:37:53.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.245+0000 7ff390c00640 1 -- 172.21.15.131:0/3614703249 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff37c042070 con 0x7ff39415bb90 2024-04-28T12:37:53.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.245+0000 7ff399c00640 1 -- 172.21.15.131:0/3614703249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39415c390 con 0x7ff39415bb90 2024-04-28T12:37:53.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.245+0000 7ff390c00640 1 -- 172.21.15.131:0/3614703249 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c030bb0 con 0x7ff39415bb90 2024-04-28T12:37:53.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.245+0000 7ff399c00640 1 -- 172.21.15.131:0/3614703249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff39415cbd0 con 0x7ff39415bb90 2024-04-28T12:37:53.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.246+0000 7ff399c00640 1 -- 172.21.15.131:0/3614703249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff394069350 con 0x7ff39415bb90 2024-04-28T12:37:53.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.247+0000 7ff390c00640 1 -- 172.21.15.131:0/3614703249 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff37c047460 con 0x7ff39415bb90 2024-04-28T12:37:53.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.248+0000 7ff390c00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff37006c8f0 0x7ff37006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.248+0000 7ff390c00640 1 -- 172.21.15.131:0/3614703249 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff37c0acbd0 con 0x7ff39415bb90 2024-04-28T12:37:53.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.249+0000 7ff393e00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff37006c8f0 0x7ff37006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.250+0000 7ff393e00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff37006c8f0 0x7ff37006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff384008890 tx=0x7ff384008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:53.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.252+0000 7ff390c00640 1 -- 172.21.15.131:0/3614703249 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff39415c390 con 0x7ff39415bb90 2024-04-28T12:37:53.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.503+0000 7ff399c00640 1 -- 172.21.15.131:0/3614703249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff39415cda0 con 0x7ff39415bb90 2024-04-28T12:37:53.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.511+0000 7ff390c00640 1 -- 172.21.15.131:0/3614703249 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7ff37c07ac20 con 0x7ff39415bb90 2024-04-28T12:37:53.510 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:53.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.515+0000 7ff399c00640 1 -- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff37006c8f0 msgr2=0x7ff37006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:53.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.515+0000 7ff399c00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff37006c8f0 0x7ff37006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff384008890 tx=0x7ff384008040 comp rx=0 tx=0).stop 2024-04-28T12:37:53.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.515+0000 7ff399c00640 1 -- 172.21.15.131:0/3614703249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff39415bb90 msgr2=0x7ff39414a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:53.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.515+0000 7ff399c00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff39415bb90 0x7ff39414a350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff37c000c00 tx=0x7ff37c0046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.515+0000 7ff399c00640 1 -- 172.21.15.131:0/3614703249 shutdown_connections 2024-04-28T12:37:53.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.515+0000 7ff399c00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff37006c8f0 0x7ff37006edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.515+0000 7ff399c00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff39415f610 0x7ff3940ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.515+0000 7ff399c00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff39415bb90 0x7ff39414a350 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.515+0000 7ff399c00640 1 --2- 172.21.15.131:0/3614703249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3940af710 0x7ff394143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.516+0000 7ff399c00640 1 -- 172.21.15.131:0/3614703249 >> 172.21.15.131:0/3614703249 conn(0x7ff394068090 msgr2=0x7ff3940b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:53.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.516+0000 7ff399c00640 1 -- 172.21.15.131:0/3614703249 shutdown_connections 2024-04-28T12:37:53.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.516+0000 7ff399c00640 1 -- 172.21.15.131:0/3614703249 wait complete. 2024-04-28T12:37:53.530 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2340731527},{"type":"v1","addr":"172.21.15.157:6801","nonce":2340731527}]},"active_addr":"172.21.15.157:6801/2340731527","active_change":"2024-04-28T12:37:06.897642+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":1044546587}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":271678319}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":3651990704}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":1205240710}]}]} 2024-04-28T12:37:53.533 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-28T12:37:53.533 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-28T12:37:53.533 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:53.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.737+0000 7f3456200640 1 Processor -- start 2024-04-28T12:37:53.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.737+0000 7f3456200640 1 -- start start 2024-04-28T12:37:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.739+0000 7f3456200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f34501467c0 0x7f3450072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.739+0000 7f3456200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3450146ba0 con 0x7f34500b4140 2024-04-28T12:37:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.739+0000 7f3456200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34500731c0 con 0x7f34500b3ce0 2024-04-28T12:37:53.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.739+0000 7f3456200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3450073dd0 con 0x7f34501467c0 2024-04-28T12:37:53.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.739+0000 7f344fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f34500b4140 0x7f345006dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33158/0 (socket says 172.21.15.131:33158) 2024-04-28T12:37:53.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.739+0000 7f344fe00640 1 -- 172.21.15.131:0/4142644291 learned_addr learned my addr 172.21.15.131:0/4142644291 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.739+0000 7f344f400640 1 --2- 172.21.15.131:0/4142644291 >> v2:172.21.15.157:3300/0 conn(0x7f34501467c0 0x7f3450072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037106881 0 0) 0x7f3450146ba0 con 0x7f34500b4140 2024-04-28T12:37:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3434003520 con 0x7f34500b4140 2024-04-28T12:37:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2320321700 0 0) 0x7f3434003520 con 0x7f34500b4140 2024-04-28T12:37:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 >> v2:172.21.15.157:3300/0 conn(0x7f34501467c0 msgr2=0x7f3450072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 --2- 172.21.15.131:0/4142644291 >> v2:172.21.15.157:3300/0 conn(0x7f34501467c0 0x7f3450072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 >> v1:172.21.15.157:6789/0 conn(0x7f34500b3ce0 legacy=0x7f34500d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34500734a0 con 0x7f34500b4140 2024-04-28T12:37:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f34480019f0 con 0x7f34500b4140 2024-04-28T12:37:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 >> v1:172.21.15.131:6789/0 conn(0x7f34500b4140 legacy=0x7f345006dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 --2- 172.21.15.131:0/4142644291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34340039b0 0x7f3434003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 --2- 172.21.15.131:0/4142644291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3434004800 0x7f3434004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 --2- 172.21.15.131:0/4142644291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34340051c0 0x7f343400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3450073dd0 con 0x7f3434004800 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f343400c060 con 0x7f34340039b0 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f343400c260 con 0x7f34340051c0 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.740+0000 7f3447e00640 1 --2- 172.21.15.131:0/4142644291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3434004800 0x7f3434004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.741+0000 7f344f400640 1 --2- 172.21.15.131:0/4142644291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34340039b0 0x7f3434003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.741+0000 7f344fe00640 1 --2- 172.21.15.131:0/4142644291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34340051c0 0x7f343400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.741+0000 7f3447e00640 1 -- 172.21.15.131:0/4142644291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34340051c0 msgr2=0x7f343400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.741+0000 7f3447e00640 1 --2- 172.21.15.131:0/4142644291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34340051c0 0x7f343400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.741+0000 7f3447e00640 1 -- 172.21.15.131:0/4142644291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34340039b0 msgr2=0x7f3434003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.741+0000 7f3447e00640 1 --2- 172.21.15.131:0/4142644291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34340039b0 0x7f3434003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.741+0000 7f3447e00640 1 -- 172.21.15.131:0/4142644291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3434003520 con 0x7f3434004800 2024-04-28T12:37:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.741+0000 7f344fe00640 1 -- 172.21.15.131:0/4142644291 reap_dead start 2024-04-28T12:37:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.742+0000 7f3447e00640 1 --2- 172.21.15.131:0/4142644291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3434004800 0x7f3434004c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3438001750 tx=0x7f3434005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd65cf267871f7ce server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.742+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438001370 con 0x7f3434004800 2024-04-28T12:37:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.742+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34380052d0 con 0x7f3434004800 2024-04-28T12:37:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.742+0000 7f344ea00640 1 -- 172.21.15.131:0/4142644291 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343802abc0 con 0x7f3434004800 2024-04-28T12:37:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.743+0000 7f3456200640 1 -- 172.21.15.131:0/4142644291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3434004800 msgr2=0x7f3434004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.743+0000 7f3456200640 1 --2- 172.21.15.131:0/4142644291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3434004800 0x7f3434004c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3438001750 tx=0x7f3434005830 comp rx=0 tx=0).stop 2024-04-28T12:37:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.743+0000 7f3456200640 1 -- 172.21.15.131:0/4142644291 shutdown_connections 2024-04-28T12:37:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.743+0000 7f3456200640 1 --2- 172.21.15.131:0/4142644291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3434004800 0x7f3434004c80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.743+0000 7f3456200640 1 -- 172.21.15.131:0/4142644291 >> 172.21.15.131:0/4142644291 conn(0x7f34500b2900 msgr2=0x7f34500b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.743+0000 7f3456200640 1 -- 172.21.15.131:0/4142644291 shutdown_connections 2024-04-28T12:37:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.743+0000 7f3456200640 1 -- 172.21.15.131:0/4142644291 wait complete. 2024-04-28T12:37:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.745+0000 7f3456200640 1 Processor -- start 2024-04-28T12:37:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.745+0000 7f3456200640 1 -- start start 2024-04-28T12:37:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3456200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3434004800 0x7f34501f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3456200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3450140a70 0x7f345015c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3456200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345013d6a0 0x7f345013db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3456200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3450072b70 con 0x7f3450140a70 2024-04-28T12:37:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3456200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3450073a60 con 0x7f3434004800 2024-04-28T12:37:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3456200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3450072cf0 con 0x7f345013d6a0 2024-04-28T12:37:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3447e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3450140a70 0x7f345015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3447e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3450140a70 0x7f345015c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45118/0 (socket says 172.21.15.131:45118) 2024-04-28T12:37:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3447e00640 1 -- 172.21.15.131:0/4190240394 learned_addr learned my addr 172.21.15.131:0/4190240394 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f344f400640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3434004800 0x7f34501f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f344fe00640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345013d6a0 0x7f345013db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3447e00640 1 -- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345013d6a0 msgr2=0x7f345013db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3447e00640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345013d6a0 0x7f345013db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3447e00640 1 -- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3434004800 msgr2=0x7f34501f3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3447e00640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3434004800 0x7f34501f3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.746+0000 7f3447e00640 1 -- 172.21.15.131:0/4190240394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345015cd40 con 0x7f3450140a70 2024-04-28T12:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.747+0000 7f3447e00640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3450140a70 0x7f345015c180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3438000c00 tx=0x7f3438001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.747+0000 7f3447400640 1 -- 172.21.15.131:0/4190240394 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343803f030 con 0x7f3450140a70 2024-04-28T12:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.747+0000 7f3447400640 1 -- 172.21.15.131:0/4190240394 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3438005480 con 0x7f3450140a70 2024-04-28T12:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.747+0000 7f3456200640 1 -- 172.21.15.131:0/4190240394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3450141020 con 0x7f3450140a70 2024-04-28T12:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.747+0000 7f3447400640 1 -- 172.21.15.131:0/4190240394 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438037040 con 0x7f3450140a70 2024-04-28T12:37:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.747+0000 7f3456200640 1 -- 172.21.15.131:0/4190240394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345013cbc0 con 0x7f3450140a70 2024-04-28T12:37:53.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.748+0000 7f3456200640 1 -- 172.21.15.131:0/4190240394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3450069350 con 0x7f3450140a70 2024-04-28T12:37:53.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.749+0000 7f3447400640 1 -- 172.21.15.131:0/4190240394 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3438042070 con 0x7f3450140a70 2024-04-28T12:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.750+0000 7f3447400640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f342c06c8f0 0x7f342c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.750+0000 7f3447400640 1 -- 172.21.15.131:0/4190240394 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f34380ab880 con 0x7f3450140a70 2024-04-28T12:37:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.751+0000 7f344f400640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f342c06c8f0 0x7f342c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:53.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.752+0000 7f344f400640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f342c06c8f0 0x7f342c06edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3440009a90 tx=0x7f3440008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:53.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.755+0000 7f3447400640 1 -- 172.21.15.131:0/4190240394 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3438079960 con 0x7f3450140a70 2024-04-28T12:37:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.959+0000 7f3456200640 1 -- 172.21.15.131:0/4190240394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34500b2690 con 0x7f3450140a70 2024-04-28T12:37:53.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.962+0000 7f3447400640 1 -- 172.21.15.131:0/4190240394 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7f3438079260 con 0x7f3450140a70 2024-04-28T12:37:53.962 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:53.962 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.965+0000 7f3456200640 1 -- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f342c06c8f0 msgr2=0x7f342c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.965+0000 7f3456200640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f342c06c8f0 0x7f342c06edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3440009a90 tx=0x7f3440008040 comp rx=0 tx=0).stop 2024-04-28T12:37:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.965+0000 7f3456200640 1 -- 172.21.15.131:0/4190240394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3450140a70 msgr2=0x7f345015c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.965+0000 7f3456200640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3450140a70 0x7f345015c180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3438000c00 tx=0x7f3438001120 comp rx=0 tx=0).stop 2024-04-28T12:37:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.965+0000 7f3456200640 1 -- 172.21.15.131:0/4190240394 shutdown_connections 2024-04-28T12:37:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.965+0000 7f3456200640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f342c06c8f0 0x7f342c06edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.965+0000 7f3456200640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f345013d6a0 0x7f345013db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.965+0000 7f3456200640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3450140a70 0x7f345015c180 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.965+0000 7f3456200640 1 --2- 172.21.15.131:0/4190240394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3434004800 0x7f34501f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.965+0000 7f3456200640 1 -- 172.21.15.131:0/4190240394 >> 172.21.15.131:0/4190240394 conn(0x7f34500b2900 msgr2=0x7f345015fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.965+0000 7f3456200640 1 -- 172.21.15.131:0/4190240394 shutdown_connections 2024-04-28T12:37:53.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:53.966+0000 7f3456200640 1 -- 172.21.15.131:0/4190240394 wait complete. 2024-04-28T12:37:53.982 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-28T12:37:53.982 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:54.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.188+0000 7f1488a00640 1 Processor -- start 2024-04-28T12:37:54.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.188+0000 7f1488a00640 1 -- start start 2024-04-28T12:37:54.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.190+0000 7f1488a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f14841628e0 0x7f1484162d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.190+0000 7f1488a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1484145710 con 0x7f1484145330 2024-04-28T12:37:54.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.190+0000 7f1488a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1484073dc0 con 0x7f14840af720 2024-04-28T12:37:54.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.190+0000 7f1488a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f14840b0250 con 0x7f14841628e0 2024-04-28T12:37:54.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.190+0000 7f1483400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1484145330 0x7f14840b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33166/0 (socket says 172.21.15.131:33166) 2024-04-28T12:37:54.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.191+0000 7f1483400640 1 -- 172.21.15.131:0/2998776259 learned_addr learned my addr 172.21.15.131:0/2998776259 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:54.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.191+0000 7f1482a00640 1 --2- 172.21.15.131:0/2998776259 >> v2:172.21.15.157:3300/0 conn(0x7f14841628e0 0x7f1484162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.191+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1687501954 0 0) 0x7f1484145710 con 0x7f1484145330 2024-04-28T12:37:54.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.191+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1468003520 con 0x7f1484145330 2024-04-28T12:37:54.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.191+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3597817036 0 0) 0x7f1468003520 con 0x7f1484145330 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.191+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 >> v2:172.21.15.157:3300/0 conn(0x7f14841628e0 msgr2=0x7f1484162d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.191+0000 7f1482000640 1 --2- 172.21.15.131:0/2998776259 >> v2:172.21.15.157:3300/0 conn(0x7f14841628e0 0x7f1484162d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 >> v1:172.21.15.157:6789/0 conn(0x7f14840af720 legacy=0x7f14840afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14840ae960 con 0x7f1484145330 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f1474001670 con 0x7f1484145330 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 >> v1:172.21.15.131:6789/0 conn(0x7f1484145330 legacy=0x7f14840b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1482000640 1 --2- 172.21.15.131:0/2998776259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14680039b0 0x7f1468003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1482000640 1 --2- 172.21.15.131:0/2998776259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1468004800 0x7f1468004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1482000640 1 --2- 172.21.15.131:0/2998776259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14680051c0 0x7f146800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f14840b0250 con 0x7f14680039b0 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f146800c090 con 0x7f1468004800 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f146800c290 con 0x7f14680051c0 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1483400640 1 --2- 172.21.15.131:0/2998776259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14680039b0 0x7f1468003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1482a00640 1 --2- 172.21.15.131:0/2998776259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1468004800 0x7f1468004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1483e00640 1 --2- 172.21.15.131:0/2998776259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14680051c0 0x7f146800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1483400640 1 -- 172.21.15.131:0/2998776259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14680051c0 msgr2=0x7f146800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1483400640 1 --2- 172.21.15.131:0/2998776259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14680051c0 0x7f146800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1483400640 1 -- 172.21.15.131:0/2998776259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1468004800 msgr2=0x7f1468004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1483400640 1 --2- 172.21.15.131:0/2998776259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1468004800 0x7f1468004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1483400640 1 -- 172.21.15.131:0/2998776259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1468003520 con 0x7f14680039b0 2024-04-28T12:37:54.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.192+0000 7f1483e00640 1 -- 172.21.15.131:0/2998776259 reap_dead start 2024-04-28T12:37:54.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.194+0000 7f1483400640 1 --2- 172.21.15.131:0/2998776259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14680039b0 0x7f1468003df0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f147400c240 tx=0x7f1468005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e15b13e6bf8ea23 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.194+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474007cf0 con 0x7f14680039b0 2024-04-28T12:37:54.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.194+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14740058d0 con 0x7f14680039b0 2024-04-28T12:37:54.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.194+0000 7f1482000640 1 -- 172.21.15.131:0/2998776259 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474005ce0 con 0x7f14680039b0 2024-04-28T12:37:54.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.195+0000 7f1488a00640 1 -- 172.21.15.131:0/2998776259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14680039b0 msgr2=0x7f1468003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.195+0000 7f1488a00640 1 --2- 172.21.15.131:0/2998776259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14680039b0 0x7f1468003df0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f147400c240 tx=0x7f1468005830 comp rx=0 tx=0).stop 2024-04-28T12:37:54.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.195+0000 7f1488a00640 1 -- 172.21.15.131:0/2998776259 shutdown_connections 2024-04-28T12:37:54.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.195+0000 7f1488a00640 1 --2- 172.21.15.131:0/2998776259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14680039b0 0x7f1468003df0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.195+0000 7f1488a00640 1 -- 172.21.15.131:0/2998776259 >> 172.21.15.131:0/2998776259 conn(0x7f1484068090 msgr2=0x7f1484068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.195+0000 7f1488a00640 1 -- 172.21.15.131:0/2998776259 shutdown_connections 2024-04-28T12:37:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.195+0000 7f1488a00640 1 -- 172.21.15.131:0/2998776259 wait complete. 2024-04-28T12:37:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.197+0000 7f1488a00640 1 Processor -- start 2024-04-28T12:37:54.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.197+0000 7f1488a00640 1 -- start start 2024-04-28T12:37:54.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.198+0000 7f1488a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14680039b0 0x7f148415c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.198+0000 7f1488a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1484161f10 0x7f14840ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.198+0000 7f1488a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14840ced30 0x7f14841e8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.198+0000 7f1488a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f14840af200 con 0x7f14840ced30 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.198+0000 7f1488a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f14840aef20 con 0x7f1484161f10 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.198+0000 7f1488a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1484162d70 con 0x7f14680039b0 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.199+0000 7f1483e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14840ced30 0x7f14841e8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.199+0000 7f1483e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14840ced30 0x7f14841e8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45146/0 (socket says 172.21.15.131:45146) 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.199+0000 7f1483e00640 1 -- 172.21.15.131:0/1753330609 learned_addr learned my addr 172.21.15.131:0/1753330609 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.199+0000 7f1482a00640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1484161f10 0x7f14840ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.199+0000 7f1483400640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14680039b0 0x7f148415c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.199+0000 7f1483e00640 1 -- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14680039b0 msgr2=0x7f148415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.199+0000 7f1483e00640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14680039b0 0x7f148415c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.199+0000 7f1483e00640 1 -- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1484161f10 msgr2=0x7f14840ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.199+0000 7f1483e00640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1484161f10 0x7f14840ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.199+0000 7f1483e00640 1 -- 172.21.15.131:0/1753330609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1484133d30 con 0x7f14840ced30 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.199+0000 7f1482a00640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1484161f10 0x7f14840ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:54.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.199+0000 7f1483e00640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14840ced30 0x7f14841e8720 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1484162cc0 tx=0x7f147c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.200+0000 7f147b400640 1 -- 172.21.15.131:0/1753330609 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c003970 con 0x7f14840ced30 2024-04-28T12:37:54.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.200+0000 7f147b400640 1 -- 172.21.15.131:0/1753330609 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147c018070 con 0x7f14840ced30 2024-04-28T12:37:54.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.200+0000 7f147b400640 1 -- 172.21.15.131:0/1753330609 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c003d70 con 0x7f14840ced30 2024-04-28T12:37:54.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.200+0000 7f1488a00640 1 -- 172.21.15.131:0/1753330609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1484134010 con 0x7f14840ced30 2024-04-28T12:37:54.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.200+0000 7f1488a00640 1 -- 172.21.15.131:0/1753330609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14841345d0 con 0x7f14840ced30 2024-04-28T12:37:54.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.201+0000 7f1488a00640 1 -- 172.21.15.131:0/1753330609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1484069350 con 0x7f14840ced30 2024-04-28T12:37:54.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.202+0000 7f147b400640 1 -- 172.21.15.131:0/1753330609 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f147c0050d0 con 0x7f14840ced30 2024-04-28T12:37:54.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.203+0000 7f147b400640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f146406c920 0x7f146406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.203+0000 7f147b400640 1 -- 172.21.15.131:0/1753330609 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f147c08bc00 con 0x7f14840ced30 2024-04-28T12:37:54.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.204+0000 7f1483400640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f146406c920 0x7f146406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.205+0000 7f1483400640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f146406c920 0x7f146406ede0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f147400c240 tx=0x7f1474009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.207+0000 7f147b400640 1 -- 172.21.15.131:0/1753330609 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f147c059c50 con 0x7f14840ced30 2024-04-28T12:37:54.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.391+0000 7f1488a00640 1 -- 172.21.15.131:0/1753330609 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f148415d290 con 0x7f14840ced30 2024-04-28T12:37:54.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.393+0000 7f147b400640 1 -- 172.21.15.131:0/1753330609 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7f147c059550 con 0x7f14840ced30 2024-04-28T12:37:54.390 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:54.390 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.396+0000 7f1488a00640 1 -- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f146406c920 msgr2=0x7f146406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.396+0000 7f1488a00640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f146406c920 0x7f146406ede0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f147400c240 tx=0x7f1474009030 comp rx=0 tx=0).stop 2024-04-28T12:37:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.396+0000 7f1488a00640 1 -- 172.21.15.131:0/1753330609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14840ced30 msgr2=0x7f14841e8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.396+0000 7f1488a00640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14840ced30 0x7f14841e8720 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1484162cc0 tx=0x7f147c00ec10 comp rx=0 tx=0).stop 2024-04-28T12:37:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.396+0000 7f1488a00640 1 -- 172.21.15.131:0/1753330609 shutdown_connections 2024-04-28T12:37:54.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.396+0000 7f1488a00640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f146406c920 0x7f146406ede0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.396+0000 7f1488a00640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14840ced30 0x7f14841e8720 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.396+0000 7f1488a00640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1484161f10 0x7f14840ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.396+0000 7f1488a00640 1 --2- 172.21.15.131:0/1753330609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14680039b0 0x7f148415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.396+0000 7f1488a00640 1 -- 172.21.15.131:0/1753330609 >> 172.21.15.131:0/1753330609 conn(0x7f1484068090 msgr2=0x7f14840b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.396+0000 7f1488a00640 1 -- 172.21.15.131:0/1753330609 shutdown_connections 2024-04-28T12:37:54.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.397+0000 7f1488a00640 1 -- 172.21.15.131:0/1753330609 wait complete. 2024-04-28T12:37:54.412 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-28T12:37:54.412 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-28T12:37:54.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-28T12:37:54.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-28T12:37:54.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-28T12:37:54.414 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-28T12:37:54.414 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-28T12:37:54.415 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-28T12:37:54.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.700+0000 7f257e200640 1 Processor -- start 2024-04-28T12:37:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.700+0000 7f257e200640 1 -- start start 2024-04-28T12:37:54.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.702+0000 7f257e200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f25700e3160 0x7f25700e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.702+0000 7f257e200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f257003c0a0 0x7f2570058050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.702+0000 7f257e200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2570058590 con 0x7f2570053cb0 2024-04-28T12:37:54.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.702+0000 7f257e200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2570007b70 con 0x7f25700e3160 2024-04-28T12:37:54.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.702+0000 7f257e200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2570058810 con 0x7f257003c0a0 2024-04-28T12:37:54.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.703+0000 7f257ce00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f25700e3160 0x7f25700e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.703+0000 7f257ce00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f25700e3160 0x7f25700e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45154/0 (socket says 172.21.15.131:45154) 2024-04-28T12:37:54.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.703+0000 7f257ce00640 1 -- 172.21.15.131:0/1738421538 learned_addr learned my addr 172.21.15.131:0/1738421538 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.703+0000 7f257ce00640 1 -- 172.21.15.131:0/1738421538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f257003c0a0 msgr2=0x7f2570058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.703+0000 7f257ce00640 1 --2- 172.21.15.131:0/1738421538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f257003c0a0 0x7f2570058050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.703+0000 7f257ce00640 1 -- 172.21.15.131:0/1738421538 >> v1:172.21.15.157:6789/0 conn(0x7f2570053cb0 legacy=0x7f2570054120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.703+0000 7f257ce00640 1 -- 172.21.15.131:0/1738421538 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2570058af0 con 0x7f25700e3160 2024-04-28T12:37:54.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.704+0000 7f44fd000640 1 Processor -- start 2024-04-28T12:37:54.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.705+0000 7f257ce00640 1 --2- 172.21.15.131:0/1738421538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25700e3160 0x7f25700e7780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f256c009870 tx=0x7f256c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2741e98213bf7e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.705+0000 7f2577400640 1 -- 172.21.15.131:0/1738421538 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c037070 con 0x7f25700e3160 2024-04-28T12:37:54.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.705+0000 7f2577400640 1 -- 172.21.15.131:0/1738421538 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f256c032440 con 0x7f25700e3160 2024-04-28T12:37:54.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.705+0000 7f2577400640 1 -- 172.21.15.131:0/1738421538 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c0327f0 con 0x7f25700e3160 2024-04-28T12:37:54.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.704+0000 7f44fd000640 1 -- start start 2024-04-28T12:37:54.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.706+0000 7f44fd000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f44f00e31c0 0x7f44f0006c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.706+0000 7f44fd000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44f0007190 0x7f44f00e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.706+0000 7f257e200640 1 -- 172.21.15.131:0/1738421538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25700e3160 msgr2=0x7f25700e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.706+0000 7f257e200640 1 --2- 172.21.15.131:0/1738421538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25700e3160 0x7f25700e7780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f256c009870 tx=0x7f256c02a230 comp rx=0 tx=0).stop 2024-04-28T12:37:54.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.706+0000 7f535b200640 1 Processor -- start 2024-04-28T12:37:54.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.707+0000 7f535b200640 1 -- start start 2024-04-28T12:37:54.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.706+0000 7f44fd000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f44f00e7360 0x7f44f00537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.706+0000 7f44fd000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f44f0057670 con 0x7f44f00e7360 2024-04-28T12:37:54.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.706+0000 7f44fd000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f44f00076f0 con 0x7f44f00e31c0 2024-04-28T12:37:54.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.706+0000 7f44fd000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44f00079d0 con 0x7f44f0007190 2024-04-28T12:37:54.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.707+0000 7f44f6a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44f0007190 0x7f44f00e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.707+0000 7f44f6a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44f0007190 0x7f44f00e6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:46526/0 (socket says 172.21.15.131:46526) 2024-04-28T12:37:54.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.707+0000 7f44f6a00640 1 -- 172.21.15.131:0/3300644027 learned_addr learned my addr 172.21.15.131:0/3300644027 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.707+0000 7f44f7400640 1 --2- 172.21.15.131:0/3300644027 >> v2:172.21.15.157:3300/0 conn(0x7f44f00e31c0 0x7f44f0006c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.708+0000 7f44f7e00640 1 --2- 172.21.15.131:0/3300644027 >> v2:172.21.15.131:3300/0 conn(0x7f44f00e7360 0x7f44f00537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.708+0000 7f257e200640 1 -- 172.21.15.131:0/1738421538 shutdown_connections 2024-04-28T12:37:54.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.708+0000 7f257e200640 1 --2- 172.21.15.131:0/1738421538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f257003c0a0 0x7f2570058050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.708+0000 7f257e200640 1 --2- 172.21.15.131:0/1738421538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25700e3160 0x7f25700e7780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.708+0000 7f257e200640 1 -- 172.21.15.131:0/1738421538 >> 172.21.15.131:0/1738421538 conn(0x7f2570058ee0 msgr2=0x7f25700592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.708+0000 7f257e200640 1 -- 172.21.15.131:0/1738421538 shutdown_connections 2024-04-28T12:37:54.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.709+0000 7f535b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f53540e30c0 0x7f5354057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.709+0000 7f535b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f535403e840 0x7f5354053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.709+0000 7f5359e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f53540e30c0 0x7f5354057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.709+0000 7f44f7e00640 1 -- 172.21.15.131:0/3300644027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44f0007190 msgr2=0x7f44f00e6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.709+0000 7f44f7e00640 1 --2- 172.21.15.131:0/3300644027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44f0007190 0x7f44f00e6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.709+0000 7f44f7e00640 1 -- 172.21.15.131:0/3300644027 >> v2:172.21.15.157:3300/0 conn(0x7f44f00e31c0 msgr2=0x7f44f0006c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.709+0000 7f44f7e00640 1 --2- 172.21.15.131:0/3300644027 >> v2:172.21.15.157:3300/0 conn(0x7f44f00e31c0 0x7f44f0006c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.709+0000 7f44f7e00640 1 -- 172.21.15.131:0/3300644027 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f0057300 con 0x7f44f00e7360 2024-04-28T12:37:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.709+0000 7f5359e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f53540e30c0 0x7f5354057d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45172/0 (socket says 172.21.15.131:45172) 2024-04-28T12:37:54.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.709+0000 7f5359400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f535403e840 0x7f5354053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.709+0000 7f5359400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f535403e840 0x7f5354053bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42440/0 (socket says 172.21.15.131:42440) 2024-04-28T12:37:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.710+0000 7f44f7e00640 1 --2- 172.21.15.131:0/3300644027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f00e7360 0x7f44f00537a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f44f802e060 tx=0x7f44f80a68c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db166ecd4f32a11f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.711+0000 7f5359e00640 1 -- 172.21.15.131:0/3317431860 learned_addr learned my addr 172.21.15.131:0/3317431860 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.711+0000 7f535b200640 1 -- 172.21.15.131:0/3317431860 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53540582d0 con 0x7f5354054130 2024-04-28T12:37:54.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.711+0000 7f535b200640 1 -- 172.21.15.131:0/3317431860 --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5354054560 con 0x7f53540e30c0 2024-04-28T12:37:54.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.711+0000 7f535b200640 1 -- 172.21.15.131:0/3317431860 --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5354058570 con 0x7f535403e840 2024-04-28T12:37:54.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.709+0000 7f257e200640 1 -- 172.21.15.131:0/1738421538 wait complete. 2024-04-28T12:37:54.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.710+0000 7f257e200640 1 Processor -- start 2024-04-28T12:37:54.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.711+0000 7f257e200640 1 -- start start 2024-04-28T12:37:54.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.711+0000 7f44f6000640 1 -- 172.21.15.131:0/3300644027 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f806e490 con 0x7f44f00e7360 2024-04-28T12:37:54.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.711+0000 7f5359e00640 1 -- 172.21.15.131:0/3317431860 >> v2:172.21.15.157:3300/0 conn(0x7f535403e840 msgr2=0x7f5354053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.711+0000 7f5359e00640 1 --2- 172.21.15.131:0/3317431860 >> v2:172.21.15.157:3300/0 conn(0x7f535403e840 0x7f5354053bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.711+0000 7f5359e00640 1 -- 172.21.15.131:0/3317431860 >> v1:172.21.15.157:6789/0 conn(0x7f5354054130 legacy=0x7f53540e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.711+0000 7f5359e00640 1 -- 172.21.15.131:0/3317431860 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5354058850 con 0x7f53540e30c0 2024-04-28T12:37:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.711+0000 7f44f6000640 1 -- 172.21.15.131:0/3300644027 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f806e630 con 0x7f44f00e7360 2024-04-28T12:37:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.711+0000 7f44f6000640 1 -- 172.21.15.131:0/3300644027 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f806ea60 con 0x7f44f00e7360 2024-04-28T12:37:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.712+0000 7f44fd000640 1 -- 172.21.15.131:0/3300644027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f00e7360 msgr2=0x7f44f00537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.712+0000 7f44fd000640 1 --2- 172.21.15.131:0/3300644027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f00e7360 0x7f44f00537a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f44f802e060 tx=0x7f44f80a68c0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.712+0000 7f257e200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f257003c0a0 0x7f25700e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.712+0000 7f257e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2570053cb0 0x7f25700de4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f257e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25700e3160 0x7f25700e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f257e200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25700e3540 con 0x7f25700e3160 2024-04-28T12:37:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f257e200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2570006640 con 0x7f2570053cb0 2024-04-28T12:37:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f257e200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2570006b60 con 0x7f257003c0a0 2024-04-28T12:37:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f44fd000640 1 -- 172.21.15.131:0/3300644027 shutdown_connections 2024-04-28T12:37:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f44fd000640 1 --2- 172.21.15.131:0/3300644027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f00e7360 0x7f44f00537a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f44fd000640 1 --2- 172.21.15.131:0/3300644027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44f0007190 0x7f44f00e6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f44fd000640 1 --2- 172.21.15.131:0/3300644027 >> v2:172.21.15.157:3300/0 conn(0x7f44f00e31c0 0x7f44f0006c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f6d42600640 1 Processor -- start 2024-04-28T12:37:54.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f6d42600640 1 -- start start 2024-04-28T12:37:54.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f44fd000640 1 -- 172.21.15.131:0/3300644027 >> 172.21.15.131:0/3300644027 conn(0x7f44f003aa30 msgr2=0x7f44f003ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f257ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f257003c0a0 0x7f25700e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f257d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25700e3160 0x7f25700e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f257d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25700e3160 0x7f25700e4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45178/0 (socket says 172.21.15.131:45178) 2024-04-28T12:37:54.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f44fd000640 1 -- 172.21.15.131:0/3300644027 shutdown_connections 2024-04-28T12:37:54.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f257d800640 1 -- 172.21.15.131:0/2560959898 learned_addr learned my addr 172.21.15.131:0/2560959898 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f2577e00640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2570053cb0 0x7f25700de4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.713+0000 7f44fd000640 1 -- 172.21.15.131:0/3300644027 wait complete. 2024-04-28T12:37:54.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.714+0000 7f5359e00640 1 --2- 172.21.15.131:0/3317431860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53540e30c0 0x7f5354057d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5354056960 tx=0x7f534c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd54d089213ebbf server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.714+0000 7f25d4800640 1 Processor -- start 2024-04-28T12:37:54.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.714+0000 7f257ce00640 1 -- 172.21.15.131:0/2560959898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2570053cb0 msgr2=0x7f25700de4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.714+0000 7f257ce00640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2570053cb0 0x7f25700de4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.714+0000 7f257ce00640 1 -- 172.21.15.131:0/2560959898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25700e3160 msgr2=0x7f25700e4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.714+0000 7f257ce00640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25700e3160 0x7f25700e4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.714+0000 7f257ce00640 1 -- 172.21.15.131:0/2560959898 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257017c640 con 0x7f257003c0a0 2024-04-28T12:37:54.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.714+0000 7f257d800640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25700e3160 0x7f25700e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:37:54.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.715+0000 7f5358a00640 1 -- 172.21.15.131:0/3317431860 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c037070 con 0x7f53540e30c0 2024-04-28T12:37:54.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.715+0000 7f5358a00640 1 -- 172.21.15.131:0/3317431860 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534c032440 con 0x7f53540e30c0 2024-04-28T12:37:54.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.715+0000 7f5358a00640 1 -- 172.21.15.131:0/3317431860 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c032810 con 0x7f53540e30c0 2024-04-28T12:37:54.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.714+0000 7f25d4800640 1 -- start start 2024-04-28T12:37:54.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f257ce00640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f257003c0a0 0x7f25700e36c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f256c031000 tx=0x7f256c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.715+0000 7f6d42600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6d3c0730e0 0x7f6d3c16e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.715+0000 7f6d42600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d3c073750 con 0x7f6d3c14cfd0 2024-04-28T12:37:54.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.715+0000 7f6d42600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d3c0b4360 con 0x7f6d3c167860 2024-04-28T12:37:54.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.715+0000 7f6d42600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6d3c0cd310 con 0x7f6d3c0730e0 2024-04-28T12:37:54.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f6d3b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6d3c14cfd0 0x7f6d3c167150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33176/0 (socket says 172.21.15.131:33176) 2024-04-28T12:37:54.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f6d3b400640 1 -- 172.21.15.131:0/17353388 learned_addr learned my addr 172.21.15.131:0/17353388 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:54.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f535b200640 1 -- 172.21.15.131:0/3317431860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53540e30c0 msgr2=0x7f5354057d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f535b200640 1 --2- 172.21.15.131:0/3317431860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53540e30c0 0x7f5354057d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5354056960 tx=0x7f534c02a2c0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f25d4800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f25cc1629a0 0x7f25cc0d33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f1220a00640 1 Processor -- start 2024-04-28T12:37:54.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f6d3be00640 1 --2- 172.21.15.131:0/17353388 >> v2:172.21.15.131:3300/0 conn(0x7f6d3c0730e0 0x7f6d3c16e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f535b200640 1 -- 172.21.15.131:0/3317431860 shutdown_connections 2024-04-28T12:37:54.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f535b200640 1 --2- 172.21.15.131:0/3317431860 >> v2:172.21.15.157:3300/0 conn(0x7f535403e840 0x7f5354053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f535b200640 1 --2- 172.21.15.131:0/3317431860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53540e30c0 0x7f5354057d90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f1220a00640 1 -- start start 2024-04-28T12:37:54.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f535b200640 1 -- 172.21.15.131:0/3317431860 >> 172.21.15.131:0/3317431860 conn(0x7f53540559e0 msgr2=0x7f5354055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f25d4800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc0b5df0 0x7f25cc0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f25d4800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25cc0d2cb0 con 0x7f25cc1625c0 2024-04-28T12:37:54.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f25d4800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f25cc0d2150 con 0x7f25cc1629a0 2024-04-28T12:37:54.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.716+0000 7f25d4800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f25cc0d2f50 con 0x7f25cc0b5df0 2024-04-28T12:37:54.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7fb172600640 1 Processor -- start 2024-04-28T12:37:54.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7f2574c00640 1 -- 172.21.15.131:0/2560959898 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c037050 con 0x7f257003c0a0 2024-04-28T12:37:54.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7f535b200640 1 -- 172.21.15.131:0/3317431860 shutdown_connections 2024-04-28T12:37:54.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7f25d1e00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f25cc1625c0 0x7f25cc0d0630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.131:42230/0 (socket says 172.21.15.131:42230) 2024-04-28T12:37:54.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7f25d1e00640 1 -- 172.21.15.131:0/3907399295 learned_addr learned my addr 172.21.15.131:0/3907399295 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:54.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7f25d1400640 1 --2- 172.21.15.131:0/3907399295 >> v2:172.21.15.157:3300/0 conn(0x7f25cc1629a0 0x7f25cc0d33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7f44fd000640 1 Processor -- start 2024-04-28T12:37:54.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.718+0000 7f535b200640 1 -- 172.21.15.131:0/3317431860 wait complete. 2024-04-28T12:37:54.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7f2574c00640 1 -- 172.21.15.131:0/2560959898 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f256c038040 con 0x7f257003c0a0 2024-04-28T12:37:54.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7f2574c00640 1 -- 172.21.15.131:0/2560959898 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c032850 con 0x7f257003c0a0 2024-04-28T12:37:54.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.718+0000 7f257e200640 1 -- 172.21.15.131:0/2560959898 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257017c920 con 0x7f257003c0a0 2024-04-28T12:37:54.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.718+0000 7f257e200640 1 -- 172.21.15.131:0/2560959898 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f257017cee0 con 0x7f257003c0a0 2024-04-28T12:37:54.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.718+0000 7f44fd000640 1 -- start start 2024-04-28T12:37:54.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7f25d2800640 1 --2- 172.21.15.131:0/3907399295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc0b5df0 0x7f25cc0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.718+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819398286 0 0) 0x7f25cc0d2cb0 con 0x7f25cc1625c0 2024-04-28T12:37:54.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.718+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25b4003520 con 0x7f25cc1625c0 2024-04-28T12:37:54.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7fb172600640 1 -- start start 2024-04-28T12:37:54.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7f6d3be00640 1 -- 172.21.15.131:0/17353388 >> v1:172.21.15.157:6789/0 conn(0x7f6d3c167860 legacy=0x7f6d3c0729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.717+0000 7f6d3be00640 1 -- 172.21.15.131:0/17353388 >> v1:172.21.15.131:6789/0 conn(0x7f6d3c14cfd0 legacy=0x7f6d3c167150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.718+0000 7f6d3be00640 1 -- 172.21.15.131:0/17353388 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c0b4640 con 0x7f6d3c0730e0 2024-04-28T12:37:54.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7f1220a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f121c1605e0 0x7f121c071c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7f1220a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f121c1609c0 0x7f121c0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7f1220a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f121c0721a0 con 0x7f121c071560 2024-04-28T12:37:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7f1220a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f121c0d3a10 con 0x7f121c1609c0 2024-04-28T12:37:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7f1220a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f121c0ce3a0 con 0x7f121c1605e0 2024-04-28T12:37:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7f6d3be00640 1 --2- 172.21.15.131:0/17353388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d3c0730e0 0x7f6d3c16e480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d280097a0 tx=0x7f6d2802a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e863078bdbc984 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 <== mon.1 v1:172.21.15.157:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2018217592 0 0) 0x7f25b4003520 con 0x7f25cc1625c0 2024-04-28T12:37:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc0b5df0 msgr2=0x7f25cc0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7f25d0a00640 1 --2- 172.21.15.131:0/3907399295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc0b5df0 0x7f25cc0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 >> v2:172.21.15.157:3300/0 conn(0x7f25cc1629a0 msgr2=0x7f25cc0d33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7f25d0a00640 1 --2- 172.21.15.131:0/3907399295 >> v2:172.21.15.157:3300/0 conn(0x7f25cc1629a0 0x7f25cc0d33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7fb172600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb16c0d3ab0 0x7fb16c0b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f1219e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f121c1605e0 0x7f121c071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f257e200640 1 -- 172.21.15.131:0/2560959898 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f257017c3f0 con 0x7f257003c0a0 2024-04-28T12:37:54.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.719+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 --> v1:172.21.15.157:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25cc0cfb70 con 0x7f25cc1625c0 2024-04-28T12:37:54.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 <== mon.1 v1:172.21.15.157:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f25c0001670 con 0x7f25cc1625c0 2024-04-28T12:37:54.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 >> v1:172.21.15.157:6789/0 conn(0x7f25cc1625c0 legacy=0x7f25cc0d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7fb172600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb16c0d2cb0 con 0x7fb16c1625c0 2024-04-28T12:37:54.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7fb16aa00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fb16c1629a0 0x7fb16c0d33a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.131:42232/0 (socket says 172.21.15.131:42232) 2024-04-28T12:37:54.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f1219e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f121c1605e0 0x7f121c071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:46544/0 (socket says 172.21.15.131:46544) 2024-04-28T12:37:54.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f1219e00640 1 -- 172.21.15.131:0/4051537311 learned_addr learned my addr 172.21.15.131:0/4051537311 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f25d0a00640 1 --2- 172.21.15.131:0/3907399295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc0b5df0 0x7f25b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f25d0a00640 1 --2- 172.21.15.131:0/3907399295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25b4003f80 0x7f25b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7fb16aa00640 1 -- 172.21.15.131:0/3387157406 learned_addr learned my addr 172.21.15.131:0/3387157406 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:54.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f25d0a00640 1 --2- 172.21.15.131:0/3907399295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b4004d40 0x7f25b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc0d2150 con 0x7f25b4004d40 2024-04-28T12:37:54.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc0d2f50 con 0x7f25b4003f80 2024-04-28T12:37:54.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f25b400b9c0 con 0x7f25cc0b5df0 2024-04-28T12:37:54.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7fb172600640 1 -- 172.21.15.131:0/3387157406 --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb16c0d2150 con 0x7fb16c1629a0 2024-04-28T12:37:54.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7fb172600640 1 -- 172.21.15.131:0/3387157406 --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb16c0d2f50 con 0x7fb16c0d3ab0 2024-04-28T12:37:54.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f1219400640 1 --2- 172.21.15.131:0/4051537311 >> v2:172.21.15.157:3300/0 conn(0x7f121c1609c0 0x7f121c0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.720+0000 7f535b200640 1 Processor -- start 2024-04-28T12:37:54.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f44fd000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44f0007190 0x7f44f0050910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f535b200640 1 -- start start 2024-04-28T12:37:54.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f1219e00640 1 -- 172.21.15.131:0/4051537311 >> v2:172.21.15.157:3300/0 conn(0x7f121c1609c0 msgr2=0x7f121c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f25d1400640 1 --2- 172.21.15.131:0/3907399295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25b4003f80 0x7f25b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f1219e00640 1 --2- 172.21.15.131:0/4051537311 >> v2:172.21.15.157:3300/0 conn(0x7f121c1609c0 0x7f121c0d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f25d2800640 1 --2- 172.21.15.131:0/3907399295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b4004d40 0x7f25b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f1219e00640 1 -- 172.21.15.131:0/4051537311 >> v1:172.21.15.131:6789/0 conn(0x7f121c071560 legacy=0x7f121c070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f6d3a000640 1 -- 172.21.15.131:0/17353388 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28037070 con 0x7f6d3c0730e0 2024-04-28T12:37:54.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f44fd000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f00e31c0 0x7f44f00df790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7fb16be00640 1 --2- 172.21.15.131:0/3387157406 >> v2:172.21.15.131:3300/0 conn(0x7fb16c0d3ab0 0x7fb16c0b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f44fd000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f00e7360 0x7f44f00dfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f44fd000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f44f003c240 con 0x7f44f00e7360 2024-04-28T12:37:54.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f44fd000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f44f003c0c0 con 0x7f44f00e31c0 2024-04-28T12:37:54.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f44fd000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f44f00e35a0 con 0x7f44f0007190 2024-04-28T12:37:54.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f44f7400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44f0007190 0x7f44f0050910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f6d3a000640 1 -- 172.21.15.131:0/17353388 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d2802acd0 con 0x7f6d3c0730e0 2024-04-28T12:37:54.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7fb16a000640 1 -- 172.21.15.131:0/3387157406 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 827364796 0 0) 0x7fb16c0d2150 con 0x7fb16c1629a0 2024-04-28T12:37:54.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f44f7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f00e7360 0x7f44f00dfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.722+0000 7f44f7400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44f0007190 0x7f44f0050910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:46576/0 (socket says 172.21.15.131:46576) 2024-04-28T12:37:54.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.722+0000 7f44f7400640 1 -- 172.21.15.131:0/1045880268 learned_addr learned my addr 172.21.15.131:0/1045880268 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.722+0000 7f25d1e00640 1 --2- 172.21.15.131:0/3907399295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc0b5df0 0x7f25b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.722+0000 7fb16a000640 1 -- 172.21.15.131:0/3387157406 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb150003520 con 0x7fb16c1629a0 2024-04-28T12:37:54.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.721+0000 7f1219e00640 1 -- 172.21.15.131:0/4051537311 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c0ce680 con 0x7f121c1605e0 2024-04-28T12:37:54.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.722+0000 7f44f6a00640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f00e31c0 0x7f44f00df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.722+0000 7f535b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f535403e840 0x7f5354051b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f535b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5354054130 0x7f53540d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.722+0000 7f6d3a000640 1 -- 172.21.15.131:0/17353388 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28037070 con 0x7f6d3c0730e0 2024-04-28T12:37:54.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f6d42600640 1 -- 172.21.15.131:0/17353388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d3c0730e0 msgr2=0x7f6d3c16e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f6d42600640 1 --2- 172.21.15.131:0/17353388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d3c0730e0 0x7f6d3c16e480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d280097a0 tx=0x7f6d2802a040 comp rx=0 tx=0).stop 2024-04-28T12:37:54.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f2f47e00640 1 Processor -- start 2024-04-28T12:37:54.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f535b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53540e30c0 0x7f53540d7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f535b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f535403bfc0 con 0x7f53540e30c0 2024-04-28T12:37:54.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f535b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53540063b0 con 0x7f535403e840 2024-04-28T12:37:54.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f535b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f53540e34a0 con 0x7f5354054130 2024-04-28T12:37:54.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.722+0000 7fb16be00640 1 -- 172.21.15.131:0/3387157406 >> v1:172.21.15.157:6789/0 conn(0x7fb16c1629a0 legacy=0x7fb16c0d33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.722+0000 7fb16be00640 1 -- 172.21.15.131:0/3387157406 >> v1:172.21.15.131:6789/0 conn(0x7fb16c1625c0 legacy=0x7fb16c0d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7fb16be00640 1 -- 172.21.15.131:0/3387157406 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c0cfb70 con 0x7fb16c0d3ab0 2024-04-28T12:37:54.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f6d42600640 1 -- 172.21.15.131:0/17353388 shutdown_connections 2024-04-28T12:37:54.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f6d42600640 1 --2- 172.21.15.131:0/17353388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d3c0730e0 0x7f6d3c16e480 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f6d42600640 1 -- 172.21.15.131:0/17353388 >> 172.21.15.131:0/17353388 conn(0x7f6d3c068940 msgr2=0x7f6d3c0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f25d1e00640 1 -- 172.21.15.131:0/3907399295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25b4003f80 msgr2=0x7f25b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f25d1e00640 1 --2- 172.21.15.131:0/3907399295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25b4003f80 0x7f25b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f25d1e00640 1 -- 172.21.15.131:0/3907399295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b4004d40 msgr2=0x7f25b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f25d1e00640 1 --2- 172.21.15.131:0/3907399295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b4004d40 0x7f25b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f5359e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f535403e840 0x7f5354051b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f25d1e00640 1 -- 172.21.15.131:0/3907399295 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b4003520 con 0x7f25cc0b5df0 2024-04-28T12:37:54.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f5359400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5354054130 0x7f53540d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f5359e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f535403e840 0x7f5354051b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42504/0 (socket says 172.21.15.131:42504) 2024-04-28T12:37:54.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f5359e00640 1 -- 172.21.15.131:0/2364999121 learned_addr learned my addr 172.21.15.131:0/2364999121 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f1219e00640 1 --2- 172.21.15.131:0/4051537311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f121c1605e0 0x7f121c071c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1208009870 tx=0x7f120802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c142d3f7107b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f44f7e00640 1 -- 172.21.15.131:0/1045880268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44f0007190 msgr2=0x7f44f0050910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f44f7e00640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f44f0007190 0x7f44f0050910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f44f7e00640 1 -- 172.21.15.131:0/1045880268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f00e31c0 msgr2=0x7f44f00df790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f44f7e00640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f44f00e31c0 0x7f44f00df790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f44f7e00640 1 -- 172.21.15.131:0/1045880268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f00503f0 con 0x7f44f00e7360 2024-04-28T12:37:54.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f6d42600640 1 -- 172.21.15.131:0/17353388 shutdown_connections 2024-04-28T12:37:54.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.723+0000 7f2f47e00640 1 -- start start 2024-04-28T12:37:54.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f1218a00640 1 -- 172.21.15.131:0/4051537311 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208037070 con 0x7f121c1605e0 2024-04-28T12:37:54.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f1218a00640 1 -- 172.21.15.131:0/4051537311 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1208032440 con 0x7f121c1605e0 2024-04-28T12:37:54.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f1218a00640 1 -- 172.21.15.131:0/4051537311 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208032810 con 0x7f121c1605e0 2024-04-28T12:37:54.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f535a800640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53540e30c0 0x7f53540d7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f2574c00640 1 -- 172.21.15.131:0/2560959898 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f256c045020 con 0x7f257003c0a0 2024-04-28T12:37:54.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f44f7e00640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f00e7360 0x7f44f00dfcd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f44f8051470 tx=0x7f44f806a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f5359e00640 1 -- 172.21.15.131:0/2364999121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5354054130 msgr2=0x7f53540d4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f5359e00640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5354054130 0x7f53540d4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f5359e00640 1 -- 172.21.15.131:0/2364999121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53540e30c0 msgr2=0x7f53540d7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f5359e00640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53540e30c0 0x7f53540d7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f5359e00640 1 -- 172.21.15.131:0/2364999121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53540e4a00 con 0x7f535403e840 2024-04-28T12:37:54.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.724+0000 7f6d42600640 1 -- 172.21.15.131:0/17353388 wait complete. 2024-04-28T12:37:54.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7f44eb400640 1 -- 172.21.15.131:0/1045880268 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f80ad8d0 con 0x7f44f00e7360 2024-04-28T12:37:54.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7fb16be00640 1 --2- 172.21.15.131:0/3387157406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c0d3ab0 0x7fb16c0b7c10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb164009d90 tx=0x7fb16402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9073c7cd0452f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7f5359e00640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f535403e840 0x7f5354051b30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f534c000c00 tx=0x7f534c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7f44fd000640 1 -- 172.21.15.131:0/1045880268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f004eba0 con 0x7f44f00e7360 2024-04-28T12:37:54.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7f44eb400640 1 -- 172.21.15.131:0/1045880268 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f80ada70 con 0x7f44f00e7360 2024-04-28T12:37:54.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7fb16a000640 1 -- 172.21.15.131:0/3387157406 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164007740 con 0x7fb16c0d3ab0 2024-04-28T12:37:54.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7fb16a000640 1 -- 172.21.15.131:0/3387157406 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1640078e0 con 0x7fb16c0d3ab0 2024-04-28T12:37:54.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7fb16a000640 1 -- 172.21.15.131:0/3387157406 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164007cb0 con 0x7fb16c0d3ab0 2024-04-28T12:37:54.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f5352000640 1 -- 172.21.15.131:0/2364999121 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c041020 con 0x7f535403e840 2024-04-28T12:37:54.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f44fd000640 1 -- 172.21.15.131:0/1045880268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f004f160 con 0x7f44f00e7360 2024-04-28T12:37:54.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f2574c00640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f255806c9c0 0x7f255806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f25d1e00640 1 --2- 172.21.15.131:0/3907399295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc0b5df0 0x7f25b40039b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f25c000c190 tx=0x7f25b4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3570038cc432e7df server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f5352000640 1 -- 172.21.15.131:0/2364999121 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534c030740 con 0x7f535403e840 2024-04-28T12:37:54.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f5352000640 1 -- 172.21.15.131:0/2364999121 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c037040 con 0x7f535403e840 2024-04-28T12:37:54.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f535b200640 1 -- 172.21.15.131:0/2364999121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53540e4ca0 con 0x7f535403e840 2024-04-28T12:37:54.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0007c80 con 0x7f25cc0b5df0 2024-04-28T12:37:54.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7f1220a00640 1 -- 172.21.15.131:0/4051537311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f121c1605e0 msgr2=0x7f121c071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7f1220a00640 1 --2- 172.21.15.131:0/4051537311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f121c1605e0 0x7f121c071c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1208009870 tx=0x7f120802a180 comp rx=0 tx=0).stop 2024-04-28T12:37:54.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7f1220a00640 1 -- 172.21.15.131:0/4051537311 shutdown_connections 2024-04-28T12:37:54.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7f1220a00640 1 --2- 172.21.15.131:0/4051537311 >> v2:172.21.15.157:3300/0 conn(0x7f121c1609c0 0x7f121c0d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7f1220a00640 1 --2- 172.21.15.131:0/4051537311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f121c1605e0 0x7f121c071c60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.725+0000 7f1220a00640 1 -- 172.21.15.131:0/4051537311 >> 172.21.15.131:0/4051537311 conn(0x7f121c0d2d50 msgr2=0x7f121c0d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f1220a00640 1 -- 172.21.15.131:0/4051537311 shutdown_connections 2024-04-28T12:37:54.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f1220a00640 1 -- 172.21.15.131:0/4051537311 wait complete. 2024-04-28T12:37:54.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f25d0a00640 1 -- 172.21.15.131:0/3907399295 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c002e070 con 0x7f25cc0b5df0 2024-04-28T12:37:54.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f2f47e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f400e31c0 0x7f2f40006c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f2f47e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2f400e7530 0x7f2f40057b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f2f47e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f400076f0 con 0x7f2f40007190 2024-04-28T12:37:54.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f2f47e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2f40057670 con 0x7f2f400e7530 2024-04-28T12:37:54.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f2f47e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2f400079d0 con 0x7f2f400e31c0 2024-04-28T12:37:54.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7f6d42600640 1 Processor -- start 2024-04-28T12:37:54.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7f25d4800640 1 -- 172.21.15.131:0/3907399295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc0b5df0 msgr2=0x7f25b40039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7f25d4800640 1 --2- 172.21.15.131:0/3907399295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc0b5df0 0x7f25b40039b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f25c000c190 tx=0x7f25b4005830 comp rx=0 tx=0).stop 2024-04-28T12:37:54.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.726+0000 7f44eb400640 1 -- 172.21.15.131:0/1045880268 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f80ad8d0 con 0x7f44f00e7360 2024-04-28T12:37:54.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7f44fd000640 1 -- 172.21.15.131:0/1045880268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44d4001030 con 0x7f44f00e7360 2024-04-28T12:37:54.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7f6d42600640 1 -- start start 2024-04-28T12:37:54.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7f2f46a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f400e31c0 0x7f2f40006c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7f25d2800640 1 -- 172.21.15.131:0/3907399295 reap_dead start 2024-04-28T12:37:54.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7f2f46a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f400e31c0 0x7f2f40006c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:46590/0 (socket says 172.21.15.131:46590) 2024-04-28T12:37:54.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7f2f46a00640 1 -- 172.21.15.131:0/244793245 learned_addr learned my addr 172.21.15.131:0/244793245 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7f25d4800640 1 -- 172.21.15.131:0/3907399295 shutdown_connections 2024-04-28T12:37:54.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7f25d4800640 1 -- 172.21.15.131:0/3907399295 >> 172.21.15.131:0/3907399295 conn(0x7f25cc0d4dc0 msgr2=0x7f25cc0d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7fb172600640 1 -- 172.21.15.131:0/3387157406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c0d3ab0 msgr2=0x7fb16c0b7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.727+0000 7fb172600640 1 --2- 172.21.15.131:0/3387157406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c0d3ab0 0x7fb16c0b7c10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb164009d90 tx=0x7fb16402a6f0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.728+0000 7fb172600640 1 -- 172.21.15.131:0/3387157406 shutdown_connections 2024-04-28T12:37:54.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.728+0000 7fb172600640 1 --2- 172.21.15.131:0/3387157406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c0d3ab0 0x7fb16c0b7c10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.728+0000 7fb172600640 1 -- 172.21.15.131:0/3387157406 >> 172.21.15.131:0/3387157406 conn(0x7fb16c0d4dc0 msgr2=0x7fb16c0d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.728+0000 7f2577e00640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f255806c9c0 0x7f255806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.728+0000 7f2f47400640 1 --2- 172.21.15.131:0/244793245 >> v2:172.21.15.157:3300/0 conn(0x7f2f400e7530 0x7f2f40057b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.728+0000 7f25d4800640 1 -- 172.21.15.131:0/3907399295 shutdown_connections 2024-04-28T12:37:54.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.728+0000 7f25d4800640 1 -- 172.21.15.131:0/3907399295 wait complete. 2024-04-28T12:37:54.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.728+0000 7f535b200640 1 -- 172.21.15.131:0/2364999121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53540e5260 con 0x7f535403e840 2024-04-28T12:37:54.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.728+0000 7f2574c00640 1 -- 172.21.15.131:0/2560959898 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f256c0aced0 con 0x7f257003c0a0 2024-04-28T12:37:54.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.728+0000 7f2574c00640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] conn(0x7f2558076130 0x7f2558078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.728+0000 7f2574c00640 1 -- 172.21.15.131:0/2560959898 --> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f256c03fe50 con 0x7f2558076130 2024-04-28T12:37:54.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.728+0000 7f2574c00640 1 -- 172.21.15.131:0/2560959898 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f256c0ad2c0 con 0x7f257003c0a0 2024-04-28T12:37:54.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.729+0000 7f257d800640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] conn(0x7f2558076130 0x7f2558078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.730+0000 7f2577e00640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f255806c9c0 0x7f255806ee80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f257004cbd0 tx=0x7f2564009110 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.729+0000 7fb172600640 1 -- 172.21.15.131:0/3387157406 shutdown_connections 2024-04-28T12:37:54.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.729+0000 7fb172600640 1 -- 172.21.15.131:0/3387157406 wait complete. 2024-04-28T12:37:54.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.729+0000 7f1220a00640 1 Processor -- start 2024-04-28T12:37:54.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.730+0000 7f535b200640 1 -- 172.21.15.131:0/2364999121 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5354007c10 con 0x7f535403e840 2024-04-28T12:37:54.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.730+0000 7f44eb400640 1 -- 172.21.15.131:0/1045880268 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44f80b4040 con 0x7f44f00e7360 2024-04-28T12:37:54.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.729+0000 7f1220a00640 1 -- start start 2024-04-28T12:37:54.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.730+0000 7f1220a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f121c071560 0x7f121c0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.730+0000 7f1220a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f121c1605e0 0x7f121c0c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.730+0000 7f1220a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f121c1609c0 0x7f121c0c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.730+0000 7f1220a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f121c0d0010 con 0x7f121c1609c0 2024-04-28T12:37:54.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.730+0000 7f1220a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f121c0cfe90 con 0x7f121c071560 2024-04-28T12:37:54.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.730+0000 7f1220a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f121c0d1550 con 0x7f121c1605e0 2024-04-28T12:37:54.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.729+0000 7f2f47400640 1 -- 172.21.15.131:0/244793245 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f400e31c0 msgr2=0x7f2f40006c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:54.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.729+0000 7f2f47400640 1 --2- 172.21.15.131:0/244793245 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f400e31c0 0x7f2f40006c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.729+0000 7f2f47400640 1 -- 172.21.15.131:0/244793245 >> v1:172.21.15.157:6789/0 conn(0x7f2f40007190 legacy=0x7f2f400e6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.729+0000 7f2f47400640 1 -- 172.21.15.131:0/244793245 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f40057300 con 0x7f2f400e7530 2024-04-28T12:37:54.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.731+0000 7f25d4800640 1 Processor -- start 2024-04-28T12:37:54.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.731+0000 7f257d800640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] conn(0x7f2558076130 0x7f2558078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.731+0000 7f25d4800640 1 -- start start 2024-04-28T12:37:54.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.731+0000 7f1219e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f121c071560 0x7f121c0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.731+0000 7f1219e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f121c071560 0x7f121c0c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42528/0 (socket says 172.21.15.131:42528) 2024-04-28T12:37:54.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.731+0000 7f1219e00640 1 -- 172.21.15.131:0/2277603630 learned_addr learned my addr 172.21.15.131:0/2277603630 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.731+0000 7f121a800640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f121c1609c0 0x7f121c0c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.731+0000 7f1219400640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f121c1605e0 0x7f121c0c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f6d42600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d3c0730e0 0x7f6d3c15f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f6d42600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3c14cfd0 0x7f6d3c1f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f6d42600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d3c167860 0x7f6d3c14b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.731+0000 7f44eb400640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f44d806caf0 0x7f44d806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f44eb400640 1 -- 172.21.15.131:0/1045880268 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f44f81285e0 con 0x7f44f00e7360 2024-04-28T12:37:54.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f121a800640 1 -- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f121c1605e0 msgr2=0x7f121c0c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f121a800640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f121c1605e0 0x7f121c0c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f6d42600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c0b5fd0 con 0x7f6d3c167860 2024-04-28T12:37:54.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f121a800640 1 -- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f121c071560 msgr2=0x7f121c0c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f121a800640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f121c071560 0x7f121c0c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f121a800640 1 -- 172.21.15.131:0/2277603630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c0c7940 con 0x7f121c1609c0 2024-04-28T12:37:54.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f6d42600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c0b5e50 con 0x7f6d3c0730e0 2024-04-28T12:37:54.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f6d42600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3c16e9c0 con 0x7f6d3c14cfd0 2024-04-28T12:37:54.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f6d3b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d3c0730e0 0x7f6d3c15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f44eb400640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] conn(0x7f44d8076270 0x7f44d80786b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.731+0000 7fb172600640 1 Processor -- start 2024-04-28T12:37:54.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7fb172600640 1 -- start start 2024-04-28T12:37:54.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f2f47400640 1 --2- 172.21.15.131:0/244793245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f400e7530 0x7f2f40057b00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2f4806b910 tx=0x7f2f480a6630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5955ca3eb977adb server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f121a800640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f121c1609c0 0x7f121c0c7400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f121400c780 tx=0x7f121400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f44eb400640 1 -- 172.21.15.131:0/1045880268 --> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f44f8076590 con 0x7f44d8076270 2024-04-28T12:37:54.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f44eb400640 1 -- 172.21.15.131:0/1045880268 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f44f81289d0 con 0x7f44f00e7360 2024-04-28T12:37:54.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f2574c00640 1 -- 172.21.15.131:0/2560959898 <== osd.1 v2:172.21.15.131:6800/703178662 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f256c03fe50 con 0x7f2558076130 2024-04-28T12:37:54.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7fb172600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb16c0d3ab0 0x7fb16c201580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7fb172600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c1625c0 0x7fb16c2051e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7fb172600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c1629a0 0x7fb16c205720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f1212000640 1 -- 172.21.15.131:0/2277603630 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214017070 con 0x7f121c1609c0 2024-04-28T12:37:54.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f1220a00640 1 -- 172.21.15.131:0/2277603630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121c0b57f0 con 0x7f121c1609c0 2024-04-28T12:37:54.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f2f45600640 1 -- 172.21.15.131:0/244793245 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4806e8a0 con 0x7f2f400e7530 2024-04-28T12:37:54.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f2f45600640 1 -- 172.21.15.131:0/244793245 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f480a6e50 con 0x7f2f400e7530 2024-04-28T12:37:54.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f1220a00640 1 -- 172.21.15.131:0/2277603630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f121c0b6b10 con 0x7f121c1609c0 2024-04-28T12:37:54.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f2f45600640 1 -- 172.21.15.131:0/244793245 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4806ed00 con 0x7f2f400e7530 2024-04-28T12:37:54.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7fb172600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c0d0d40 con 0x7fb16c1629a0 2024-04-28T12:37:54.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7fb172600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c0d0130 con 0x7fb16c1625c0 2024-04-28T12:37:54.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f44f6a00640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] conn(0x7f44d8076270 0x7f44d80786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7fb172600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb16c0b5df0 con 0x7fb16c0d3ab0 2024-04-28T12:37:54.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7fb16be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c1629a0 0x7fb16c205720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7fb16be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c1629a0 0x7fb16c205720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45258/0 (socket says 172.21.15.131:45258) 2024-04-28T12:37:54.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7fb16be00640 1 -- 172.21.15.131:0/2323986449 learned_addr learned my addr 172.21.15.131:0/2323986449 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7fb16aa00640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c1625c0 0x7fb16c2051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f1212000640 1 -- 172.21.15.131:0/2277603630 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1214003d80 con 0x7f121c1609c0 2024-04-28T12:37:54.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f1212000640 1 -- 172.21.15.131:0/2277603630 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12140052b0 con 0x7f121c1609c0 2024-04-28T12:37:54.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f25d4800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25cc0b5df0 0x7f25cc0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7f25d1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25cc0b5df0 0x7f25cc0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7f25d4800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25cc154fa0 0x7f25cc148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7f25d4800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc155420 0x7f25cc1493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7fb16be00640 1 -- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb16c0d3ab0 msgr2=0x7fb16c201580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-28T12:37:54.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7fb16be00640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb16c0d3ab0 0x7fb16c201580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7f2f47e00640 1 -- 172.21.15.131:0/244793245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f400e7530 msgr2=0x7f2f40057b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7f2f47e00640 1 --2- 172.21.15.131:0/244793245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f400e7530 0x7f2f40057b00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2f4806b910 tx=0x7f2f480a6630 comp rx=0 tx=0).stop 2024-04-28T12:37:54.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f2f47e00640 1 -- 172.21.15.131:0/244793245 shutdown_connections 2024-04-28T12:37:54.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f2f47e00640 1 --2- 172.21.15.131:0/244793245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f400e7530 0x7f2f40057b00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f2f47e00640 1 --2- 172.21.15.131:0/244793245 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f400e31c0 0x7f2f40006c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f2f47e00640 1 -- 172.21.15.131:0/244793245 >> 172.21.15.131:0/244793245 conn(0x7f2f4003aa30 msgr2=0x7f2f4003ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7fb16be00640 1 -- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c1625c0 msgr2=0x7fb16c2051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7fb16be00640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c1625c0 0x7fb16c2051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7fb16be00640 1 -- 172.21.15.131:0/2323986449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c201ac0 con 0x7fb16c1629a0 2024-04-28T12:37:54.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7fb16aa00640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb16c1625c0 0x7fb16c2051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:54.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f1220a00640 1 -- 172.21.15.131:0/2277603630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f121c069350 con 0x7f121c1609c0 2024-04-28T12:37:54.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f5352000640 1 -- 172.21.15.131:0/2364999121 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f534c044020 con 0x7f535403e840 2024-04-28T12:37:54.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f44f6a00640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] conn(0x7f44d8076270 0x7f44d80786b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f2f47e00640 1 -- 172.21.15.131:0/244793245 shutdown_connections 2024-04-28T12:37:54.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7fb16be00640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c1629a0 0x7fb16c205720 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb164009eb0 tx=0x7fb164004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f2f47e00640 1 -- 172.21.15.131:0/244793245 wait complete. 2024-04-28T12:37:54.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.736+0000 7f44f7400640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f44d806caf0 0x7f44d806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.736+0000 7fb163400640 1 -- 172.21.15.131:0/2323986449 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164040070 con 0x7fb16c1629a0 2024-04-28T12:37:54.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.736+0000 7fb163400640 1 -- 172.21.15.131:0/2323986449 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb164031b40 con 0x7fb16c1629a0 2024-04-28T12:37:54.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.736+0000 7f44eb400640 1 -- 172.21.15.131:0/1045880268 <== osd.2 v2:172.21.15.131:6808/768553345 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44f8076590 con 0x7f44d8076270 2024-04-28T12:37:54.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.736+0000 7fb172600640 1 -- 172.21.15.131:0/2323986449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16c0cda90 con 0x7fb16c1629a0 2024-04-28T12:37:54.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.736+0000 7fb172600640 1 -- 172.21.15.131:0/2323986449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16c0ce050 con 0x7fb16c1629a0 2024-04-28T12:37:54.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.737+0000 7fb163400640 1 -- 172.21.15.131:0/2323986449 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164045460 con 0x7fb16c1629a0 2024-04-28T12:37:54.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.737+0000 7f5352000640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f533c06c8f0 0x7f533c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.737+0000 7f5352000640 1 -- 172.21.15.131:0/2364999121 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f534c0accb0 con 0x7f535403e840 2024-04-28T12:37:54.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.737+0000 7f5352000640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] conn(0x7f533c075ff0 0x7f533c078430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.737+0000 7f5352000640 1 -- 172.21.15.131:0/2364999121 --> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f534c001d30 con 0x7f533c075ff0 2024-04-28T12:37:54.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.737+0000 7f5352000640 1 -- 172.21.15.131:0/2364999121 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f534c0ad060 con 0x7f535403e840 2024-04-28T12:37:54.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.738+0000 7f535a800640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] conn(0x7f533c075ff0 0x7f533c078430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.737+0000 7fb172600640 1 -- 172.21.15.131:0/2323986449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb144001030 con 0x7fb16c1629a0 2024-04-28T12:37:54.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.737+0000 7f2f47e00640 1 Processor -- start 2024-04-28T12:37:54.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.738+0000 7f2f47e00640 1 -- start start 2024-04-28T12:37:54.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f535a800640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] conn(0x7f533c075ff0 0x7f533c078430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7f25d4800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc0d0d40 con 0x7f25cc0b5df0 2024-04-28T12:37:54.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7f25d4800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc0d0130 con 0x7f25cc154fa0 2024-04-28T12:37:54.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7f25d4800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f25cc0d58e0 con 0x7f25cc155420 2024-04-28T12:37:54.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7f25d1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25cc0b5df0 0x7f25cc0ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45260/0 (socket says 172.21.15.131:45260) 2024-04-28T12:37:54.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7f25d1e00640 1 -- 172.21.15.131:0/377649647 learned_addr learned my addr 172.21.15.131:0/377649647 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f25d2800640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc155420 0x7f25cc1493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f25d1e00640 1 -- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc155420 msgr2=0x7f25cc1493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f25d1e00640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25cc155420 0x7f25cc1493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.735+0000 7f25d1e00640 1 -- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25cc154fa0 msgr2=0x7f25cc148eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:37:54.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f25d1e00640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25cc154fa0 0x7f25cc148eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f2f47e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f40007190 0x7f2f4004ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f2f47e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f400e31c0 0x7f2f400d82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f2f47e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f400e7530 0x7f2f4004c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f25d1e00640 1 -- 172.21.15.131:0/377649647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25cc0cd710 con 0x7f25cc0b5df0 2024-04-28T12:37:54.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f2f47e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4003c240 con 0x7f2f400e31c0 2024-04-28T12:37:54.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f2f47e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4003c0c0 con 0x7f2f40007190 2024-04-28T12:37:54.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f2f47e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2f400e35a0 con 0x7f2f400e7530 2024-04-28T12:37:54.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.737+0000 7f1212000640 1 -- 172.21.15.131:0/2277603630 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f121401a070 con 0x7f121c1609c0 2024-04-28T12:37:54.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f1212000640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f11fc06c8f0 0x7f11fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f1212000640 1 -- 172.21.15.131:0/2277603630 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f121408bc80 con 0x7f121c1609c0 2024-04-28T12:37:54.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f2f46000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f400e31c0 0x7f2f400d82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.732+0000 7f6d3b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d3c0730e0 0x7f6d3c15f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42542/0 (socket says 172.21.15.131:42542) 2024-04-28T12:37:54.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7f6d3b400640 1 -- 172.21.15.131:0/1712153260 learned_addr learned my addr 172.21.15.131:0/1712153260 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.733+0000 7f6d3be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d3c167860 0x7f6d3c14b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f1212000640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] conn(0x7f11fc075ff0 0x7f11fc078430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f1212000640 1 -- 172.21.15.131:0/2277603630 --> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11fc078b00 con 0x7f11fc075ff0 2024-04-28T12:37:54.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f1212000640 1 -- 172.21.15.131:0/2277603630 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f121408c030 con 0x7f121c1609c0 2024-04-28T12:37:54.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f2f46000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f400e31c0 0x7f2f400d82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45266/0 (socket says 172.21.15.131:45266) 2024-04-28T12:37:54.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.739+0000 7f2f46000640 1 -- 172.21.15.131:0/2155616999 learned_addr learned my addr 172.21.15.131:0/2155616999 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:54.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.734+0000 7f6d3aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3c14cfd0 0x7f6d3c1f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f6d3be00640 1 -- 172.21.15.131:0/1712153260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3c14cfd0 msgr2=0x7f6d3c1f9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f6d3be00640 1 --2- 172.21.15.131:0/1712153260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d3c14cfd0 0x7f6d3c1f9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f6d3be00640 1 -- 172.21.15.131:0/1712153260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d3c0730e0 msgr2=0x7f6d3c15f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f6d3be00640 1 --2- 172.21.15.131:0/1712153260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d3c0730e0 0x7f6d3c15f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f6d3be00640 1 -- 172.21.15.131:0/1712153260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c1fa1e0 con 0x7f6d3c167860 2024-04-28T12:37:54.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f2f46000640 1 -- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f400e7530 msgr2=0x7f2f4004c9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-28T12:37:54.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f2f46000640 1 --2- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f400e7530 0x7f2f4004c9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f2f46000640 1 -- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f40007190 msgr2=0x7f2f4004ef40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-28T12:37:54.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f2f46000640 1 --2- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f40007190 0x7f2f4004ef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f2f46000640 1 -- 172.21.15.131:0/2155616999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4004f6c0 con 0x7f2f400e31c0 2024-04-28T12:37:54.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f1219e00640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f11fc06c8f0 0x7f11fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f5352000640 1 -- 172.21.15.131:0/2364999121 <== osd.0 v2:172.21.15.131:6813/1049412824 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f534c001d30 con 0x7f533c075ff0 2024-04-28T12:37:54.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.740+0000 7f6d3be00640 1 --2- 172.21.15.131:0/1712153260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d3c167860 0x7f6d3c14b740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0b30a0 tx=0x7f6d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.741+0000 7f1219400640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] conn(0x7f11fc075ff0 0x7f11fc078430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.741+0000 7f5359400640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f533c06c8f0 0x7f533c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.741+0000 7f25d1e00640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25cc0b5df0 0x7f25cc0ccee0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f25c0007fb0 tx=0x7f25c0001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.741+0000 7f2f46000640 1 --2- 172.21.15.131:0/2155616999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f400e31c0 0x7f2f400d82e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400c400 tx=0x7f2f3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.741+0000 7f6d33400640 1 -- 172.21.15.131:0/1712153260 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28037070 con 0x7f6d3c167860 2024-04-28T12:37:54.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.741+0000 7f6d33400640 1 -- 172.21.15.131:0/1712153260 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d28004030 con 0x7f6d3c167860 2024-04-28T12:37:54.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.741+0000 7f6d33400640 1 -- 172.21.15.131:0/1712153260 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28004420 con 0x7f6d3c167860 2024-04-28T12:37:54.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.741+0000 7f6d42600640 1 -- 172.21.15.131:0/1712153260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c148fb0 con 0x7f6d3c167860 2024-04-28T12:37:54.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.742+0000 7fb163400640 1 -- 172.21.15.131:0/2323986449 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb164039040 con 0x7fb16c1629a0 2024-04-28T12:37:54.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.742+0000 7f6d42600640 1 -- 172.21.15.131:0/1712153260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c148680 con 0x7f6d3c167860 2024-04-28T12:37:54.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.742+0000 7f1219400640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] conn(0x7f11fc075ff0 0x7f11fc078430 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.742+0000 7f5359400640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f533c06c8f0 0x7f533c06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f535c07bf40 tx=0x7f535c08e040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.743+0000 7f44f7400640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f44d806caf0 0x7f44d806efb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009c00 tx=0x7f44ec009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.742+0000 7f25c6000640 1 -- 172.21.15.131:0/377649647 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0007c80 con 0x7f25cc0b5df0 2024-04-28T12:37:54.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.743+0000 7f25c6000640 1 -- 172.21.15.131:0/377649647 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25c0033470 con 0x7f25cc0b5df0 2024-04-28T12:37:54.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.743+0000 7f25c6000640 1 -- 172.21.15.131:0/377649647 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0001bc0 con 0x7f25cc0b5df0 2024-04-28T12:37:54.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.743+0000 7f1219e00640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f11fc06c8f0 0x7f11fc06edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12080090b0 tx=0x7f120802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.743+0000 7f6d42600640 1 -- 172.21.15.131:0/1712153260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d3c069350 con 0x7f6d3c167860 2024-04-28T12:37:54.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.744+0000 7f25d4800640 1 -- 172.21.15.131:0/377649647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25cc0cd9f0 con 0x7f25cc0b5df0 2024-04-28T12:37:54.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.744+0000 7f25d4800640 1 -- 172.21.15.131:0/377649647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25cc0cdfb0 con 0x7f25cc0b5df0 2024-04-28T12:37:54.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.744+0000 7fb163400640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb14c06c9c0 0x7fb14c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.743+0000 7f2f3ea00640 1 -- 172.21.15.131:0/2155616999 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34017070 con 0x7f2f400e31c0 2024-04-28T12:37:54.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.743+0000 7f2f3ea00640 1 -- 172.21.15.131:0/2155616999 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f340052d0 con 0x7f2f400e31c0 2024-04-28T12:37:54.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.743+0000 7f2f3ea00640 1 -- 172.21.15.131:0/2155616999 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34003830 con 0x7f2f400e31c0 2024-04-28T12:37:54.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.744+0000 7fb163400640 1 -- 172.21.15.131:0/2323986449 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb1640adc30 con 0x7fb16c1629a0 2024-04-28T12:37:54.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.744+0000 7f2f47e00640 1 -- 172.21.15.131:0/2155616999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f400501d0 con 0x7f2f400e31c0 2024-04-28T12:37:54.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.744+0000 7f2f47e00640 1 -- 172.21.15.131:0/2155616999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f40050790 con 0x7f2f400e31c0 2024-04-28T12:37:54.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.745+0000 7f25bb400640 1 -- 172.21.15.131:0/377649647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f25a4001030 con 0x7f25cc0b5df0 2024-04-28T12:37:54.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.746+0000 7f1212000640 1 -- 172.21.15.131:0/2277603630 <== osd.7 v2:172.21.15.157:6821/2246043568 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11fc078b00 con 0x7f11fc075ff0 2024-04-28T12:37:54.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.745+0000 7fb163400640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] conn(0x7fb14c076140 0x7fb14c078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.746+0000 7fb163400640 1 -- 172.21.15.131:0/2323986449 --> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb16402ee50 con 0x7fb14c076140 2024-04-28T12:37:54.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.746+0000 7fb163400640 1 -- 172.21.15.131:0/2323986449 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb1640ae020 con 0x7fb16c1629a0 2024-04-28T12:37:54.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.745+0000 7f6d33400640 1 -- 172.21.15.131:0/1712153260 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d2803c680 con 0x7f6d3c167860 2024-04-28T12:37:54.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.746+0000 7fb16b400640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb14c06c9c0 0x7fb14c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.745+0000 7f2f47e00640 1 -- 172.21.15.131:0/2155616999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f400057e0 con 0x7f2f400e31c0 2024-04-28T12:37:54.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.748+0000 7f6d33400640 1 --2- 172.21.15.131:0/1712153260 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6d1c06c940 0x7f6d1c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.748+0000 7fb16aa00640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] conn(0x7fb14c076140 0x7fb14c078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.748+0000 7f6d33400640 1 -- 172.21.15.131:0/1712153260 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6d280aca40 con 0x7f6d3c167860 2024-04-28T12:37:54.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.749+0000 7f6d33400640 1 --2- 172.21.15.131:0/1712153260 >> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] conn(0x7f6d1c0760b0 0x7f6d1c0784f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.749+0000 7f6d33400640 1 -- 172.21.15.131:0/1712153260 --> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d2800af30 con 0x7f6d1c0760b0 2024-04-28T12:37:54.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.749+0000 7fb16aa00640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] conn(0x7fb14c076140 0x7fb14c078580 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.749+0000 7fb16b400640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb14c06c9c0 0x7fb14c06ee80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb15c000f60 tx=0x7fb15c0079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.748+0000 7f2f3ea00640 1 -- 172.21.15.131:0/2155616999 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f3401a070 con 0x7f2f400e31c0 2024-04-28T12:37:54.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.749+0000 7f2f3ea00640 1 --2- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2f2806c940 0x7f2f2806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.749+0000 7f6d33400640 1 -- 172.21.15.131:0/1712153260 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6d280acdf0 con 0x7f6d3c167860 2024-04-28T12:37:54.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.750+0000 7f2f3ea00640 1 -- 172.21.15.131:0/2155616999 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2f34056ce0 con 0x7f2f400e31c0 2024-04-28T12:37:54.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.750+0000 7f6d3aa00640 1 --2- 172.21.15.131:0/1712153260 >> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] conn(0x7f6d1c0760b0 0x7f6d1c0784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.750+0000 7f2f3ea00640 1 --2- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] conn(0x7f2f28076d00 0x7f2f280791a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.750+0000 7f2f3ea00640 1 -- 172.21.15.131:0/2155616999 --> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f28079870 con 0x7f2f28076d00 2024-04-28T12:37:54.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.750+0000 7f2f3ea00640 1 -- 172.21.15.131:0/2155616999 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2f34055670 con 0x7f2f400e31c0 2024-04-28T12:37:54.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.750+0000 7f2f46a00640 1 --2- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2f2806c940 0x7f2f2806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.750+0000 7f6d3b400640 1 --2- 172.21.15.131:0/1712153260 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6d1c06c940 0x7f6d1c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.751+0000 7f6d3aa00640 1 --2- 172.21.15.131:0/1712153260 >> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] conn(0x7f6d1c0760b0 0x7f6d1c0784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.752+0000 7f2f46a00640 1 --2- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2f2806c940 0x7f2f2806ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2f380096f0 tx=0x7f2f38008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.752+0000 7f6d3b400640 1 --2- 172.21.15.131:0/1712153260 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6d1c06c940 0x7f6d1c06ee00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6d34001a70 tx=0x7f6d34001a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.750+0000 7f25c6000640 1 -- 172.21.15.131:0/377649647 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25c003e070 con 0x7f25cc0b5df0 2024-04-28T12:37:54.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.752+0000 7f25c6000640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f25b006c8f0 0x7f25b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.752+0000 7f25c6000640 1 -- 172.21.15.131:0/377649647 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f25c00abf70 con 0x7f25cc0b5df0 2024-04-28T12:37:54.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.752+0000 7f25c6000640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] conn(0x7f25b0076070 0x7f25b00784b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:54.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.752+0000 7f25c6000640 1 -- 172.21.15.131:0/377649647 --> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25b0078b80 con 0x7f25b0076070 2024-04-28T12:37:54.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.752+0000 7f25c6000640 1 -- 172.21.15.131:0/377649647 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f25c00ac360 con 0x7f25cc0b5df0 2024-04-28T12:37:54.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.752+0000 7f6d33400640 1 -- 172.21.15.131:0/1712153260 <== osd.3 v2:172.21.15.131:6812/708415318 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d2800af30 con 0x7f6d1c0760b0 2024-04-28T12:37:54.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.752+0000 7fb163400640 1 -- 172.21.15.131:0/2323986449 <== osd.5 v2:172.21.15.157:6818/2721118630 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb16402ee50 con 0x7fb14c076140 2024-04-28T12:37:54.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.753+0000 7f25d1400640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f25b006c8f0 0x7f25b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.753+0000 7f25d2800640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] conn(0x7f25b0076070 0x7f25b00784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.754+0000 7f2f47400640 1 --2- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] conn(0x7f2f28076d00 0x7f2f280791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.754+0000 7f25d1400640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f25b006c8f0 0x7f25b006edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f25cc154a40 tx=0x7f25bc009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.754+0000 7f25d2800640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] conn(0x7f25b0076070 0x7f25b00784b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.755+0000 7f2f47400640 1 --2- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] conn(0x7f2f28076d00 0x7f2f280791a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:54.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.756+0000 7f25c6000640 1 -- 172.21.15.131:0/377649647 <== osd.4 v2:172.21.15.157:6802/4053377461 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25b0078b80 con 0x7f25b0076070 2024-04-28T12:37:54.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.757+0000 7f2f3ea00640 1 -- 172.21.15.131:0/2155616999 <== osd.6 v2:172.21.15.157:6810/2594435113 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2f28079870 con 0x7f2f28076d00 2024-04-28T12:37:54.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.763+0000 7f535b200640 1 -- 172.21.15.131:0/2364999121 --> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f53540057e0 con 0x7f533c075ff0 2024-04-28T12:37:54.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.764+0000 7f5352000640 1 -- 172.21.15.131:0/2364999121 <== osd.0 v2:172.21.15.131:6813/1049412824 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f53540057e0 con 0x7f533c075ff0 2024-04-28T12:37:54.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.765+0000 7f535b200640 1 -- 172.21.15.131:0/2364999121 >> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] conn(0x7f533c075ff0 msgr2=0x7f533c078430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.765+0000 7f535b200640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] conn(0x7f533c075ff0 0x7f533c078430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.767+0000 7f44fd000640 1 -- 172.21.15.131:0/1045880268 --> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f44d4002dd0 con 0x7f44d8076270 2024-04-28T12:37:54.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.767+0000 7f535b200640 1 -- 172.21.15.131:0/2364999121 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f533c06c8f0 msgr2=0x7f533c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.767+0000 7f535b200640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f533c06c8f0 0x7f533c06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f535c07bf40 tx=0x7f535c08e040 comp rx=0 tx=0).stop 2024-04-28T12:37:54.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.767+0000 7f535b200640 1 -- 172.21.15.131:0/2364999121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f535403e840 msgr2=0x7f5354051b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.767+0000 7f535b200640 1 --2- 172.21.15.131:0/2364999121 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f535403e840 0x7f5354051b30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f534c000c00 tx=0x7f534c0046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.767+0000 7f535a800640 1 -- 172.21.15.131:0/2364999121 reap_dead start 2024-04-28T12:37:54.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.767+0000 7f44eb400640 1 -- 172.21.15.131:0/1045880268 <== osd.2 v2:172.21.15.131:6808/768553345 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f44d4002dd0 con 0x7f44d8076270 2024-04-28T12:37:54.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.768+0000 7f44fd000640 1 -- 172.21.15.131:0/1045880268 >> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] conn(0x7f44d8076270 msgr2=0x7f44d80786b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.768+0000 7f44fd000640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] conn(0x7f44d8076270 0x7f44d80786b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.768+0000 7f535b200640 1 -- 172.21.15.131:0/2364999121 shutdown_connections 2024-04-28T12:37:54.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.768+0000 7f535b200640 1 -- 172.21.15.131:0/2364999121 >> 172.21.15.131:0/2364999121 conn(0x7f53540559e0 msgr2=0x7f53540380e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.768+0000 7f535b200640 1 -- 172.21.15.131:0/2364999121 shutdown_connections 2024-04-28T12:37:54.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.768+0000 7f44fd000640 1 -- 172.21.15.131:0/1045880268 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f44d806caf0 msgr2=0x7f44d806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.768+0000 7f44fd000640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f44d806caf0 0x7f44d806efb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009c00 tx=0x7f44ec009190 comp rx=0 tx=0).stop 2024-04-28T12:37:54.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.768+0000 7f44fd000640 1 -- 172.21.15.131:0/1045880268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f00e7360 msgr2=0x7f44f00dfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.768+0000 7f44fd000640 1 --2- 172.21.15.131:0/1045880268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44f00e7360 0x7f44f00dfcd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f44f8051470 tx=0x7f44f806a370 comp rx=0 tx=0).stop 2024-04-28T12:37:54.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.769+0000 7f535b200640 1 -- 172.21.15.131:0/2364999121 wait complete. 2024-04-28T12:37:54.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.769+0000 7f44f7e00640 1 -- 172.21.15.131:0/1045880268 reap_dead start 2024-04-28T12:37:54.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.769+0000 7f44fd000640 1 -- 172.21.15.131:0/1045880268 shutdown_connections 2024-04-28T12:37:54.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.769+0000 7f44fd000640 1 -- 172.21.15.131:0/1045880268 >> 172.21.15.131:0/1045880268 conn(0x7f44f003aa30 msgr2=0x7f44f003b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.769+0000 7f44fd000640 1 -- 172.21.15.131:0/1045880268 shutdown_connections 2024-04-28T12:37:54.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.769+0000 7f44fd000640 1 -- 172.21.15.131:0/1045880268 wait complete. 2024-04-28T12:37:54.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.770+0000 7f257e200640 1 -- 172.21.15.131:0/2560959898 --> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2570005390 con 0x7f2558076130 2024-04-28T12:37:54.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.772+0000 7f2574c00640 1 -- 172.21.15.131:0/2560959898 <== osd.1 v2:172.21.15.131:6800/703178662 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2570005390 con 0x7f2558076130 2024-04-28T12:37:54.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.773+0000 7f257e200640 1 -- 172.21.15.131:0/2560959898 >> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] conn(0x7f2558076130 msgr2=0x7f2558078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.773+0000 7f257e200640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] conn(0x7f2558076130 0x7f2558078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.774+0000 7f257e200640 1 -- 172.21.15.131:0/2560959898 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f255806c9c0 msgr2=0x7f255806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.774+0000 7f257e200640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f255806c9c0 0x7f255806ee80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f257004cbd0 tx=0x7f2564009110 comp rx=0 tx=0).stop 2024-04-28T12:37:54.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.774+0000 7f257e200640 1 -- 172.21.15.131:0/2560959898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f257003c0a0 msgr2=0x7f25700e36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.774+0000 7f257e200640 1 --2- 172.21.15.131:0/2560959898 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f257003c0a0 0x7f25700e36c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f256c031000 tx=0x7f256c0046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.774+0000 7f257d800640 1 -- 172.21.15.131:0/2560959898 reap_dead start 2024-04-28T12:37:54.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.774+0000 7f257e200640 1 -- 172.21.15.131:0/2560959898 shutdown_connections 2024-04-28T12:37:54.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.774+0000 7f257e200640 1 -- 172.21.15.131:0/2560959898 >> 172.21.15.131:0/2560959898 conn(0x7f2570058ee0 msgr2=0x7f2570037da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.775+0000 7f257e200640 1 -- 172.21.15.131:0/2560959898 shutdown_connections 2024-04-28T12:37:54.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.775+0000 7f257e200640 1 -- 172.21.15.131:0/2560959898 wait complete. 2024-04-28T12:37:54.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.781+0000 7f1220a00640 1 -- 172.21.15.131:0/2277603630 --> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f121c0b7280 con 0x7f11fc075ff0 2024-04-28T12:37:54.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.782+0000 7f1212000640 1 -- 172.21.15.131:0/2277603630 <== osd.7 v2:172.21.15.157:6821/2246043568 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f121c0b7280 con 0x7f11fc075ff0 2024-04-28T12:37:54.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.783+0000 7f1220a00640 1 -- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] conn(0x7f11fc075ff0 msgr2=0x7f11fc078430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.783+0000 7f1220a00640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] conn(0x7f11fc075ff0 0x7f11fc078430 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.785+0000 7f1220a00640 1 -- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f11fc06c8f0 msgr2=0x7f11fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.785+0000 7f1220a00640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f11fc06c8f0 0x7f11fc06edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12080090b0 tx=0x7f120802e000 comp rx=0 tx=0).stop 2024-04-28T12:37:54.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.785+0000 7f1220a00640 1 -- 172.21.15.131:0/2277603630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f121c1609c0 msgr2=0x7f121c0c7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.785+0000 7f1220a00640 1 --2- 172.21.15.131:0/2277603630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f121c1609c0 0x7f121c0c7400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f121400c780 tx=0x7f121400cc40 comp rx=0 tx=0).stop 2024-04-28T12:37:54.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.785+0000 7f25bb400640 1 -- 172.21.15.131:0/377649647 --> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f25a4002dd0 con 0x7f25b0076070 2024-04-28T12:37:54.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.786+0000 7f6d42600640 1 -- 172.21.15.131:0/1712153260 --> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d3c0b3fa0 con 0x7f6d1c0760b0 2024-04-28T12:37:54.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.786+0000 7f121a800640 1 -- 172.21.15.131:0/2277603630 reap_dead start 2024-04-28T12:37:54.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.787+0000 7f1220a00640 1 -- 172.21.15.131:0/2277603630 shutdown_connections 2024-04-28T12:37:54.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.787+0000 7f1220a00640 1 -- 172.21.15.131:0/2277603630 >> 172.21.15.131:0/2277603630 conn(0x7f121c0d2d50 msgr2=0x7f121c0cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.787+0000 7f25c6000640 1 -- 172.21.15.131:0/377649647 <== osd.4 v2:172.21.15.157:6802/4053377461 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f25a4002dd0 con 0x7f25b0076070 2024-04-28T12:37:54.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.787+0000 7f1220a00640 1 -- 172.21.15.131:0/2277603630 shutdown_connections 2024-04-28T12:37:54.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.787+0000 7f1220a00640 1 -- 172.21.15.131:0/2277603630 wait complete. 2024-04-28T12:37:54.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.788+0000 7f25bb400640 1 -- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] conn(0x7f25b0076070 msgr2=0x7f25b00784b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.788+0000 7f25bb400640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] conn(0x7f25b0076070 0x7f25b00784b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.787+0000 7f2f47e00640 1 -- 172.21.15.131:0/2155616999 --> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f4000f170 con 0x7f2f28076d00 2024-04-28T12:37:54.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.788+0000 7f2f3ea00640 1 -- 172.21.15.131:0/2155616999 <== osd.6 v2:172.21.15.157:6810/2594435113 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2f4000f170 con 0x7f2f28076d00 2024-04-28T12:37:54.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.789+0000 7f25bb400640 1 -- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f25b006c8f0 msgr2=0x7f25b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.789+0000 7f6d33400640 1 -- 172.21.15.131:0/1712153260 <== osd.3 v2:172.21.15.131:6812/708415318 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6d3c0b3fa0 con 0x7f6d1c0760b0 2024-04-28T12:37:54.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.789+0000 7f25bb400640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f25b006c8f0 0x7f25b006edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f25cc154a40 tx=0x7f25bc009190 comp rx=0 tx=0).stop 2024-04-28T12:37:54.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.789+0000 7f25bb400640 1 -- 172.21.15.131:0/377649647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25cc0b5df0 msgr2=0x7f25cc0ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.789+0000 7f25bb400640 1 --2- 172.21.15.131:0/377649647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25cc0b5df0 0x7f25cc0ccee0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f25c0007fb0 tx=0x7f25c0001970 comp rx=0 tx=0).stop 2024-04-28T12:37:54.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.789+0000 7f25d2800640 1 -- 172.21.15.131:0/377649647 reap_dead start 2024-04-28T12:37:54.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.789+0000 7f2f47e00640 1 -- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] conn(0x7f2f28076d00 msgr2=0x7f2f280791a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.789+0000 7f2f47e00640 1 --2- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] conn(0x7f2f28076d00 0x7f2f280791a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.790+0000 7f6d42600640 1 -- 172.21.15.131:0/1712153260 >> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] conn(0x7f6d1c0760b0 msgr2=0x7f6d1c0784f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.790+0000 7f6d42600640 1 --2- 172.21.15.131:0/1712153260 >> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] conn(0x7f6d1c0760b0 0x7f6d1c0784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.790+0000 7f25bb400640 1 -- 172.21.15.131:0/377649647 shutdown_connections 2024-04-28T12:37:54.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.790+0000 7f25bb400640 1 -- 172.21.15.131:0/377649647 >> 172.21.15.131:0/377649647 conn(0x7f25cc0d4dc0 msgr2=0x7f25cc15fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.790+0000 7fb172600640 1 -- 172.21.15.131:0/2323986449 --> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb144002dd0 con 0x7fb14c076140 2024-04-28T12:37:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.790+0000 7f2f47e00640 1 -- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2f2806c940 msgr2=0x7f2f2806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.790+0000 7f2f47e00640 1 --2- 172.21.15.131:0/2155616999 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2f2806c940 0x7f2f2806ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2f380096f0 tx=0x7f2f38008040 comp rx=0 tx=0).stop 2024-04-28T12:37:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.790+0000 7f2f47e00640 1 -- 172.21.15.131:0/2155616999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f400e31c0 msgr2=0x7f2f400d82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.790+0000 7f2f47e00640 1 --2- 172.21.15.131:0/2155616999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f400e31c0 0x7f2f400d82e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400c400 tx=0x7f2f3400c8c0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.790+0000 7f2f47400640 1 -- 172.21.15.131:0/2155616999 reap_dead start 2024-04-28T12:37:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.790+0000 7f2f47e00640 1 -- 172.21.15.131:0/2155616999 shutdown_connections 2024-04-28T12:37:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.790+0000 7f2f47e00640 1 -- 172.21.15.131:0/2155616999 >> 172.21.15.131:0/2155616999 conn(0x7f2f4003aa30 msgr2=0x7f2f400d9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.791+0000 7f6d42600640 1 -- 172.21.15.131:0/1712153260 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6d1c06c940 msgr2=0x7f6d1c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.791+0000 7f6d42600640 1 --2- 172.21.15.131:0/1712153260 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6d1c06c940 0x7f6d1c06ee00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6d34001a70 tx=0x7f6d34001a00 comp rx=0 tx=0).stop 2024-04-28T12:37:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.791+0000 7f6d42600640 1 -- 172.21.15.131:0/1712153260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d3c167860 msgr2=0x7f6d3c14b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.791+0000 7f6d42600640 1 --2- 172.21.15.131:0/1712153260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d3c167860 0x7f6d3c14b740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0b30a0 tx=0x7f6d280046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.791+0000 7f2f47e00640 1 -- 172.21.15.131:0/2155616999 shutdown_connections 2024-04-28T12:37:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.791+0000 7f6d3be00640 1 -- 172.21.15.131:0/1712153260 reap_dead start 2024-04-28T12:37:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.791+0000 7f6d42600640 1 -- 172.21.15.131:0/1712153260 shutdown_connections 2024-04-28T12:37:54.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.791+0000 7f6d42600640 1 -- 172.21.15.131:0/1712153260 >> 172.21.15.131:0/1712153260 conn(0x7f6d3c068940 msgr2=0x7f6d3c0b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.791+0000 7f2f47e00640 1 -- 172.21.15.131:0/2155616999 wait complete. 2024-04-28T12:37:54.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.791+0000 7f25bb400640 1 -- 172.21.15.131:0/377649647 shutdown_connections 2024-04-28T12:37:54.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.791+0000 7fb163400640 1 -- 172.21.15.131:0/2323986449 <== osd.5 v2:172.21.15.157:6818/2721118630 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb144002dd0 con 0x7fb14c076140 2024-04-28T12:37:54.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.792+0000 7f25bb400640 1 -- 172.21.15.131:0/377649647 wait complete. 2024-04-28T12:37:54.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.791+0000 7f6d42600640 1 -- 172.21.15.131:0/1712153260 shutdown_connections 2024-04-28T12:37:54.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.792+0000 7fb172600640 1 -- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] conn(0x7fb14c076140 msgr2=0x7fb14c078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.792+0000 7fb172600640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] conn(0x7fb14c076140 0x7fb14c078580 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.792+0000 7f6d42600640 1 -- 172.21.15.131:0/1712153260 wait complete. 2024-04-28T12:37:54.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.792+0000 7fb172600640 1 -- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb14c06c9c0 msgr2=0x7fb14c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.792+0000 7fb172600640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb14c06c9c0 0x7fb14c06ee80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb15c000f60 tx=0x7fb15c0079b0 comp rx=0 tx=0).stop 2024-04-28T12:37:54.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.792+0000 7fb172600640 1 -- 172.21.15.131:0/2323986449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c1629a0 msgr2=0x7fb16c205720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:54.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.792+0000 7fb172600640 1 --2- 172.21.15.131:0/2323986449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb16c1629a0 0x7fb16c205720 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb164009eb0 tx=0x7fb164004a40 comp rx=0 tx=0).stop 2024-04-28T12:37:54.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.792+0000 7fb16be00640 1 -- 172.21.15.131:0/2323986449 reap_dead start 2024-04-28T12:37:54.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.793+0000 7fb172600640 1 -- 172.21.15.131:0/2323986449 shutdown_connections 2024-04-28T12:37:54.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.793+0000 7fb172600640 1 -- 172.21.15.131:0/2323986449 >> 172.21.15.131:0/2323986449 conn(0x7fb16c0d4dc0 msgr2=0x7fb16c0c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:54.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.793+0000 7fb172600640 1 -- 172.21.15.131:0/2323986449 shutdown_connections 2024-04-28T12:37:54.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:54.793+0000 7fb172600640 1 -- 172.21.15.131:0/2323986449 wait complete. 2024-04-28T12:37:54.827 INFO:teuthology.orchestra.run.smithi131.stdout:55834574858 2024-04-28T12:37:54.827 INFO:teuthology.orchestra.run.smithi131.stdout:55834574857 2024-04-28T12:37:54.828 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-28T12:37:54.828 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-28T12:37:54.829 INFO:teuthology.orchestra.run.smithi131.stdout:55834574859 2024-04-28T12:37:54.830 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-28T12:37:54.830 INFO:teuthology.orchestra.run.smithi131.stdout:55834574857 2024-04-28T12:37:54.831 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-28T12:37:54.831 INFO:teuthology.orchestra.run.smithi131.stdout:55834574857 2024-04-28T12:37:54.832 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-28T12:37:54.832 INFO:teuthology.orchestra.run.smithi131.stdout:55834574859 2024-04-28T12:37:54.833 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-28T12:37:54.833 INFO:teuthology.orchestra.run.smithi131.stdout:55834574859 2024-04-28T12:37:54.834 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-28T12:37:54.835 INFO:teuthology.orchestra.run.smithi131.stdout:55834574858 2024-04-28T12:37:54.835 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-28T12:37:55.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.148+0000 7fb773200640 1 Processor -- start 2024-04-28T12:37:55.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.149+0000 7fb773200640 1 -- start start 2024-04-28T12:37:55.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.151+0000 7fb773200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb7740d3480 0x7fb7740c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.151+0000 7fb773200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb7740d1d90 0x7fb774162170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.151+0000 7fb773200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7740cf860 con 0x7fb7740b5ca0 2024-04-28T12:37:55.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.151+0000 7fb773200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb7740d3090 con 0x7fb7740d3480 2024-04-28T12:37:55.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.151+0000 7fb773200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb7740d5b50 con 0x7fb7740d1d90 2024-04-28T12:37:55.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.151+0000 7fb772800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb7740b5ca0 0x7fb7740cffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33210/0 (socket says 172.21.15.131:33210) 2024-04-28T12:37:55.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.151+0000 7fb772800640 1 -- 172.21.15.131:0/197434485 learned_addr learned my addr 172.21.15.131:0/197434485 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:55.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.152+0000 7fb771400640 1 --2- 172.21.15.131:0/197434485 >> v2:172.21.15.157:3300/0 conn(0x7fb7740d1d90 0x7fb774162170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.153+0000 7fb771e00640 1 --2- 172.21.15.131:0/197434485 >> v2:172.21.15.131:3300/0 conn(0x7fb7740d3480 0x7fb7740c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.154+0000 7fb771400640 1 -- 172.21.15.131:0/197434485 >> v2:172.21.15.131:3300/0 conn(0x7fb7740d3480 msgr2=0x7fb7740c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:55.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.154+0000 7fb771400640 1 --2- 172.21.15.131:0/197434485 >> v2:172.21.15.131:3300/0 conn(0x7fb7740d3480 0x7fb7740c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.154+0000 7fb771400640 1 -- 172.21.15.131:0/197434485 >> v1:172.21.15.131:6789/0 conn(0x7fb7740b5ca0 legacy=0x7fb7740cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.154+0000 7fb771400640 1 -- 172.21.15.131:0/197434485 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7740cfb00 con 0x7fb7740d1d90 2024-04-28T12:37:55.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.156+0000 7f8a4e800640 1 Processor -- start 2024-04-28T12:37:55.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.156+0000 7f8a4e800640 1 -- start start 2024-04-28T12:37:55.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.156+0000 7fb771400640 1 --2- 172.21.15.131:0/197434485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7740d1d90 0x7fb774162170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb77408d520 tx=0x7fb76002a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fc776d26177cb40 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.157+0000 7fb770a00640 1 -- 172.21.15.131:0/197434485 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760037070 con 0x7fb7740d1d90 2024-04-28T12:37:55.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.157+0000 7fb770a00640 1 -- 172.21.15.131:0/197434485 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb760032440 con 0x7fb7740d1d90 2024-04-28T12:37:55.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.157+0000 7fb770a00640 1 -- 172.21.15.131:0/197434485 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760037070 con 0x7fb7740d1d90 2024-04-28T12:37:55.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.158+0000 7fb773200640 1 -- 172.21.15.131:0/197434485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7740d1d90 msgr2=0x7fb774162170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.158+0000 7f8a4e800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8a480b71a0 0x7f8a480d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.158+0000 7f8a4e800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8a480b7580 0x7f8a4815e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.158+0000 7fb773200640 1 --2- 172.21.15.131:0/197434485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7740d1d90 0x7fb774162170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb77408d520 tx=0x7fb76002a1e0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.159+0000 7f8a47e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8a4815ef00 0x7f8a48162da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33216/0 (socket says 172.21.15.131:33216) 2024-04-28T12:37:55.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.159+0000 7f8a4e800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a480d3a10 con 0x7f8a4815ef00 2024-04-28T12:37:55.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.159+0000 7f8a4e800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f8a480c8830 con 0x7f8a480b71a0 2024-04-28T12:37:55.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.159+0000 7f8a4e800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8a480d3cb0 con 0x7f8a480b7580 2024-04-28T12:37:55.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.159+0000 7f8a47e00640 1 -- 172.21.15.131:0/2160095387 learned_addr learned my addr 172.21.15.131:0/2160095387 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:55.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.159+0000 7f8a46a00640 1 --2- 172.21.15.131:0/2160095387 >> v2:172.21.15.157:3300/0 conn(0x7f8a480b7580 0x7f8a4815e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.159+0000 7f8a47400640 1 --2- 172.21.15.131:0/2160095387 >> v2:172.21.15.131:3300/0 conn(0x7f8a480b71a0 0x7f8a480d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.160+0000 7f8a47400640 1 -- 172.21.15.131:0/2160095387 >> v2:172.21.15.157:3300/0 conn(0x7f8a480b7580 msgr2=0x7f8a4815e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:55.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.160+0000 7f8a47400640 1 --2- 172.21.15.131:0/2160095387 >> v2:172.21.15.157:3300/0 conn(0x7f8a480b7580 0x7f8a4815e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.160+0000 7f8a47400640 1 -- 172.21.15.131:0/2160095387 >> v1:172.21.15.131:6789/0 conn(0x7f8a4815ef00 legacy=0x7f8a48162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.160+0000 7f8a47400640 1 -- 172.21.15.131:0/2160095387 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a480c8af0 con 0x7f8a480b71a0 2024-04-28T12:37:55.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.160+0000 7fb773200640 1 -- 172.21.15.131:0/197434485 shutdown_connections 2024-04-28T12:37:55.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.160+0000 7fb773200640 1 --2- 172.21.15.131:0/197434485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7740d1d90 0x7fb774162170 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.160+0000 7fb773200640 1 --2- 172.21.15.131:0/197434485 >> v2:172.21.15.131:3300/0 conn(0x7fb7740d3480 0x7fb7740c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.160+0000 7fb773200640 1 -- 172.21.15.131:0/197434485 >> 172.21.15.131:0/197434485 conn(0x7fb7740d4c50 msgr2=0x7fb7740d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.160+0000 7f5501400640 1 Processor -- start 2024-04-28T12:37:55.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.161+0000 7fb773200640 1 -- 172.21.15.131:0/197434485 shutdown_connections 2024-04-28T12:37:55.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.162+0000 7f8a47400640 1 --2- 172.21.15.131:0/2160095387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a480b71a0 0x7f8a480d34d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8a4808d0b0 tx=0x7f8a4002a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d9f87afb2c8276 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.161+0000 7f5501400640 1 -- start start 2024-04-28T12:37:55.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.163+0000 7f8a46000640 1 -- 172.21.15.131:0/2160095387 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40007740 con 0x7f8a480b71a0 2024-04-28T12:37:55.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.163+0000 7f5501400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc072000 0x7f54fc15b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.163+0000 7f5501400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54fc068bb0 0x7f54fc0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.163+0000 7f8a46000640 1 -- 172.21.15.131:0/2160095387 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a400078e0 con 0x7f8a480b71a0 2024-04-28T12:37:55.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.163+0000 7f5501400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54fc15c2f0 con 0x7f54fc151e40 2024-04-28T12:37:55.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.163+0000 7f5501400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f54fc06e1d0 con 0x7f54fc068bb0 2024-04-28T12:37:55.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.163+0000 7f5501400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f54fc0b46a0 con 0x7f54fc072000 2024-04-28T12:37:55.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.161+0000 7fb773200640 1 -- 172.21.15.131:0/197434485 wait complete. 2024-04-28T12:37:55.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.163+0000 7fb773200640 1 Processor -- start 2024-04-28T12:37:55.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.164+0000 7f5500a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54fc068bb0 0x7f54fc0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.164+0000 7f54fb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc072000 0x7f54fc15b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.164+0000 7f5500a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54fc068bb0 0x7f54fc0b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42604/0 (socket says 172.21.15.131:42604) 2024-04-28T12:37:55.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.164+0000 7f5500a00640 1 -- 172.21.15.131:0/3384401647 learned_addr learned my addr 172.21.15.131:0/3384401647 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.164+0000 7f8a46000640 1 -- 172.21.15.131:0/2160095387 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40007740 con 0x7f8a480b71a0 2024-04-28T12:37:55.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.164+0000 7fb773200640 1 -- start start 2024-04-28T12:37:55.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.165+0000 7f8a4e800640 1 -- 172.21.15.131:0/2160095387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a480b71a0 msgr2=0x7f8a480d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.165+0000 7f8a4e800640 1 --2- 172.21.15.131:0/2160095387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a480b71a0 0x7f8a480d34d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8a4808d0b0 tx=0x7f8a4002a4e0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.165+0000 7f5500a00640 1 -- 172.21.15.131:0/3384401647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc072000 msgr2=0x7f54fc15b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.165+0000 7f5500a00640 1 --2- 172.21.15.131:0/3384401647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc072000 0x7f54fc15b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.166+0000 7f8a4e800640 1 -- 172.21.15.131:0/2160095387 shutdown_connections 2024-04-28T12:37:55.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.166+0000 7f8a4e800640 1 --2- 172.21.15.131:0/2160095387 >> v2:172.21.15.157:3300/0 conn(0x7f8a480b7580 0x7f8a4815e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.165+0000 7f5500a00640 1 -- 172.21.15.131:0/3384401647 >> v1:172.21.15.131:6789/0 conn(0x7f54fc151e40 legacy=0x7f54fc06dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.166+0000 7f8a4e800640 1 --2- 172.21.15.131:0/2160095387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a480b71a0 0x7f8a480d34d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.166+0000 7f8a4e800640 1 -- 172.21.15.131:0/2160095387 >> 172.21.15.131:0/2160095387 conn(0x7f8a480d4d70 msgr2=0x7f8a480d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.165+0000 7f5500a00640 1 -- 172.21.15.131:0/3384401647 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc152220 con 0x7f54fc068bb0 2024-04-28T12:37:55.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.166+0000 7f54fb400640 1 --2- 172.21.15.131:0/3384401647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc072000 0x7f54fc15b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:37:55.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.166+0000 7f8a4e800640 1 -- 172.21.15.131:0/2160095387 shutdown_connections 2024-04-28T12:37:55.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.167+0000 7fb773200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7740b5ca0 0x7fb774155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.167+0000 7f8a4e800640 1 -- 172.21.15.131:0/2160095387 wait complete. 2024-04-28T12:37:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.168+0000 7f5500a00640 1 --2- 172.21.15.131:0/3384401647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc068bb0 0x7f54fc0b5130 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f54fc1523f0 tx=0x7f54e802c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c97debc7ae136f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.168+0000 7f4c2b200640 1 Processor -- start 2024-04-28T12:37:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.168+0000 7f54faa00640 1 -- 172.21.15.131:0/3384401647 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8038070 con 0x7f54fc068bb0 2024-04-28T12:37:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.167+0000 7fb773200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7740d1d90 0x7fb7740ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.167+0000 7fb773200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7740d3480 0x7fb774150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.167+0000 7fb773200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb7740d5770 con 0x7fb7740d3480 2024-04-28T12:37:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.167+0000 7fb773200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb7740d2db0 con 0x7fb7740b5ca0 2024-04-28T12:37:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.167+0000 7fb773200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb7740d58f0 con 0x7fb7740d1d90 2024-04-28T12:37:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.168+0000 7fb771400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7740d1d90 0x7fb7740ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.168+0000 7fb772800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7740d3480 0x7fb774150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.168+0000 7fb772800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7740d3480 0x7fb774150f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45290/0 (socket says 172.21.15.131:45290) 2024-04-28T12:37:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.168+0000 7fb772800640 1 -- 172.21.15.131:0/1029735975 learned_addr learned my addr 172.21.15.131:0/1029735975 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.168+0000 7f4c2b200640 1 -- start start 2024-04-28T12:37:55.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.169+0000 7f54faa00640 1 -- 172.21.15.131:0/3384401647 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e8031710 con 0x7f54fc068bb0 2024-04-28T12:37:55.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.169+0000 7fb772800640 1 -- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7740d1d90 msgr2=0x7fb7740ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.169+0000 7fb772800640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7740d1d90 0x7fb7740ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.169+0000 7fb772800640 1 -- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7740b5ca0 msgr2=0x7fb774155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.169+0000 7fb771e00640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7740b5ca0 0x7fb774155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.169+0000 7fb772800640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7740b5ca0 0x7fb774155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.169+0000 7fb772800640 1 -- 172.21.15.131:0/1029735975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb774070560 con 0x7fb7740d3480 2024-04-28T12:37:55.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.169+0000 7f54faa00640 1 -- 172.21.15.131:0/3384401647 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8031ae0 con 0x7f54fc068bb0 2024-04-28T12:37:55.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.169+0000 7fb771e00640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7740b5ca0 0x7fb774155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:55.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.170+0000 7fb772800640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7740d3480 0x7fb774150f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00caa0 tx=0x7fb76c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.170+0000 7f8a4e800640 1 Processor -- start 2024-04-28T12:37:55.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.170+0000 7f5501400640 1 -- 172.21.15.131:0/3384401647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc068bb0 msgr2=0x7f54fc0b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.170+0000 7f5501400640 1 --2- 172.21.15.131:0/3384401647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc068bb0 0x7f54fc0b5130 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f54fc1523f0 tx=0x7f54e802c3d0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.170+0000 7fb76a000640 1 -- 172.21.15.131:0/1029735975 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c017070 con 0x7fb7740d3480 2024-04-28T12:37:55.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.170+0000 7fb76a000640 1 -- 172.21.15.131:0/1029735975 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76c00ac00 con 0x7fb7740d3480 2024-04-28T12:37:55.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.170+0000 7fb76a000640 1 -- 172.21.15.131:0/1029735975 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c012620 con 0x7fb7740d3480 2024-04-28T12:37:55.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.170+0000 7f5501400640 1 -- 172.21.15.131:0/3384401647 shutdown_connections 2024-04-28T12:37:55.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.170+0000 7f5501400640 1 --2- 172.21.15.131:0/3384401647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc068bb0 0x7f54fc0b5130 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.170+0000 7f5501400640 1 --2- 172.21.15.131:0/3384401647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc072000 0x7f54fc15b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.170+0000 7f5501400640 1 -- 172.21.15.131:0/3384401647 >> 172.21.15.131:0/3384401647 conn(0x7f54fc068090 msgr2=0x7f54fc0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7f5501400640 1 -- 172.21.15.131:0/3384401647 shutdown_connections 2024-04-28T12:37:55.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7f4c2b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c240b7580 0x7f4c2415e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7f4c2b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4c2415ef00 0x7f4c2415f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7f4c2b200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c240c8830 con 0x7f4c240b71a0 2024-04-28T12:37:55.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7f4c2b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4c240d3cb0 con 0x7f4c2415ef00 2024-04-28T12:37:55.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7f4c2b200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4c240c8af0 con 0x7f4c240b7580 2024-04-28T12:37:55.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7fb773200640 1 -- 172.21.15.131:0/1029735975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb774155280 con 0x7fb7740d3480 2024-04-28T12:37:55.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7fb773200640 1 -- 172.21.15.131:0/1029735975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb774155840 con 0x7fb7740d3480 2024-04-28T12:37:55.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.171+0000 7f8a4e800640 1 -- start start 2024-04-28T12:37:55.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.173+0000 7f4c23400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4c240b7580 0x7f4c2415e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7f8a4e800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a480b71a0 0x7f8a481f9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7f8a4e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a480b7580 0x7f8a480ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7f8a4e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a4815ef00 0x7f8a480cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.173+0000 7f4c23e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4c240b71a0 0x7f4c240d34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33226/0 (socket says 172.21.15.131:33226) 2024-04-28T12:37:55.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7f8a4e800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4815f330 con 0x7f8a4815ef00 2024-04-28T12:37:55.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.173+0000 7f8a4e800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8a480c90b0 con 0x7f8a480b71a0 2024-04-28T12:37:55.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.173+0000 7f8a4e800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8a4815f4b0 con 0x7f8a480b7580 2024-04-28T12:37:55.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.173+0000 7f4c23e00640 1 -- 172.21.15.131:0/234682544 learned_addr learned my addr 172.21.15.131:0/234682544 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:55.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.173+0000 7f4c29200640 1 --2- 172.21.15.131:0/234682544 >> v2:172.21.15.131:3300/0 conn(0x7f4c2415ef00 0x7f4c2415f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.174+0000 7f8a47e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a4815ef00 0x7f8a480cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.174+0000 7f8a47e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a4815ef00 0x7f8a480cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45314/0 (socket says 172.21.15.131:45314) 2024-04-28T12:37:55.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.174+0000 7f8a47e00640 1 -- 172.21.15.131:0/665906274 learned_addr learned my addr 172.21.15.131:0/665906274 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.174+0000 7f4c23400640 1 -- 172.21.15.131:0/234682544 >> v2:172.21.15.131:3300/0 conn(0x7f4c2415ef00 msgr2=0x7f4c2415f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:55.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.174+0000 7f4c23400640 1 --2- 172.21.15.131:0/234682544 >> v2:172.21.15.131:3300/0 conn(0x7f4c2415ef00 0x7f4c2415f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.174+0000 7f4c23400640 1 -- 172.21.15.131:0/234682544 >> v1:172.21.15.131:6789/0 conn(0x7f4c240b71a0 legacy=0x7f4c240d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.174+0000 7f4c23400640 1 -- 172.21.15.131:0/234682544 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c240c8dd0 con 0x7f4c240b7580 2024-04-28T12:37:55.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.174+0000 7fac46a00640 1 Processor -- start 2024-04-28T12:37:55.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.172+0000 7f5501400640 1 -- 172.21.15.131:0/3384401647 wait complete. 2024-04-28T12:37:55.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.174+0000 7f5501400640 1 Processor -- start 2024-04-28T12:37:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.175+0000 7f8a46a00640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a480b7580 0x7f8a480ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.175+0000 7f8a47400640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a480b71a0 0x7f8a481f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.175+0000 7f8a47e00640 1 -- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a480b7580 msgr2=0x7f8a480ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.175+0000 7f8a47e00640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a480b7580 0x7f8a480ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.175+0000 7f8a47e00640 1 -- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a480b71a0 msgr2=0x7f8a481f9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.175+0000 7f8a47e00640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a480b71a0 0x7f8a481f9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.175+0000 7f8a47e00640 1 -- 172.21.15.131:0/665906274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a480cec10 con 0x7f8a4815ef00 2024-04-28T12:37:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.176+0000 7f8a47e00640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a4815ef00 0x7f8a480cce00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800c560 tx=0x7f8a3800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.176+0000 7f8a3f400640 1 -- 172.21.15.131:0/665906274 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38018070 con 0x7f8a4815ef00 2024-04-28T12:37:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.176+0000 7f8a3f400640 1 -- 172.21.15.131:0/665906274 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a380074e0 con 0x7f8a4815ef00 2024-04-28T12:37:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.176+0000 7f8a4e800640 1 -- 172.21.15.131:0/665906274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a480cd630 con 0x7f8a4815ef00 2024-04-28T12:37:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.176+0000 7f8a3f400640 1 -- 172.21.15.131:0/665906274 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38007690 con 0x7f8a4815ef00 2024-04-28T12:37:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.176+0000 7fac46a00640 1 -- start start 2024-04-28T12:37:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.177+0000 7fac46a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fac400d1620 0x7fac400d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.177+0000 7fac46a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fac400d1a00 0x7fac400b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.177+0000 7fac46a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400b50b0 0x7fac400b5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.177+0000 7fac46a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fac400d0ce0 con 0x7fac400d1a00 2024-04-28T12:37:55.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.177+0000 7fac46a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fac400d0480 con 0x7fac400d1620 2024-04-28T12:37:55.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.177+0000 7fac46a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac400d0720 con 0x7fac400b50b0 2024-04-28T12:37:55.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.176+0000 7fb76a000640 1 -- 172.21.15.131:0/1029735975 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb76c01a070 con 0x7fb7740d3480 2024-04-28T12:37:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.178+0000 7fac46000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400b50b0 0x7fac400b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.178+0000 7fac46000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400b50b0 0x7fac400b5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:46640/0 (socket says 172.21.15.131:46640) 2024-04-28T12:37:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.178+0000 7fac46000640 1 -- 172.21.15.131:0/2829251935 learned_addr learned my addr 172.21.15.131:0/2829251935 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.178+0000 7fac46000640 1 -- 172.21.15.131:0/2829251935 >> v2:172.21.15.157:3300/0 conn(0x7fac400d1620 msgr2=0x7fac400d5750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-28T12:37:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.178+0000 7fac46000640 1 --2- 172.21.15.131:0/2829251935 >> v2:172.21.15.157:3300/0 conn(0x7fac400d1620 0x7fac400d5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.178+0000 7fac46000640 1 -- 172.21.15.131:0/2829251935 >> v2:172.21.15.131:3300/0 conn(0x7fac400d1a00 msgr2=0x7fac400b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.178+0000 7fac46000640 1 --2- 172.21.15.131:0/2829251935 >> v2:172.21.15.131:3300/0 conn(0x7fac400d1a00 0x7fac400b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.179+0000 7fac46000640 1 -- 172.21.15.131:0/2829251935 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac400d0a00 con 0x7fac400b50b0 2024-04-28T12:37:55.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.179+0000 7f4c23400640 1 --2- 172.21.15.131:0/234682544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c240b7580 0x7f4c2415e9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4c240d4a80 tx=0x7f4c1402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fcc553840adf4b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.178+0000 7f5501400640 1 -- start start 2024-04-28T12:37:55.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.179+0000 7fb76a000640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb75406c9c0 0x7fb75406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.179+0000 7f4c22a00640 1 -- 172.21.15.131:0/234682544 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14037070 con 0x7f4c240b7580 2024-04-28T12:37:55.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.179+0000 7f4c22a00640 1 -- 172.21.15.131:0/234682544 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c14032440 con 0x7f4c240b7580 2024-04-28T12:37:55.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.179+0000 7f4c22a00640 1 -- 172.21.15.131:0/234682544 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1402ae60 con 0x7f4c240b7580 2024-04-28T12:37:55.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.179+0000 7fb76a000640 1 -- 172.21.15.131:0/1029735975 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb76c08bb00 con 0x7fb7740d3480 2024-04-28T12:37:55.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.176+0000 7fb773200640 1 -- 172.21.15.131:0/1029735975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb774161c40 con 0x7fb7740d3480 2024-04-28T12:37:55.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.180+0000 7f5501400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc068bb0 0x7f54fc154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.180+0000 7f5501400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc072000 0x7f54fc150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.180+0000 7f5501400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54fc151e40 0x7f54fc1fc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.180+0000 7f5501400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc15bd30 con 0x7f54fc151e40 2024-04-28T12:37:55.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.180+0000 7f5501400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc15ba50 con 0x7f54fc068bb0 2024-04-28T12:37:55.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.180+0000 7f5501400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f54fc15c010 con 0x7f54fc072000 2024-04-28T12:37:55.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.180+0000 7fac46000640 1 --2- 172.21.15.131:0/2829251935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400b50b0 0x7fac400b5540 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fac4008d330 tx=0x7fac3402a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ccf40be620c9ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.180+0000 7f8a4e800640 1 -- 172.21.15.131:0/665906274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a480cdbf0 con 0x7f8a4815ef00 2024-04-28T12:37:55.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7f4c2b200640 1 -- 172.21.15.131:0/234682544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c240b7580 msgr2=0x7f4c2415e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7f4c2b200640 1 --2- 172.21.15.131:0/234682544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c240b7580 0x7f4c2415e9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4c240d4a80 tx=0x7f4c1402a110 comp rx=0 tx=0).stop 2024-04-28T12:37:55.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7f54fbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc068bb0 0x7f54fc154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7f54fbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc068bb0 0x7f54fc154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42650/0 (socket says 172.21.15.131:42650) 2024-04-28T12:37:55.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7f54fbe00640 1 -- 172.21.15.131:0/2974703319 learned_addr learned my addr 172.21.15.131:0/2974703319 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7f5500a00640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54fc151e40 0x7f54fc1fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.182+0000 7f54fb400640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc072000 0x7f54fc150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.182+0000 7f54fb400640 1 -- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc068bb0 msgr2=0x7f54fc154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.182+0000 7f54fb400640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc068bb0 0x7f54fc154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7fb771e00640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb75406c9c0 0x7fb75406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7f4c2b200640 1 -- 172.21.15.131:0/234682544 shutdown_connections 2024-04-28T12:37:55.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7f4c2b200640 1 --2- 172.21.15.131:0/234682544 >> v2:172.21.15.131:3300/0 conn(0x7f4c2415ef00 0x7f4c2415f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7f4c2b200640 1 --2- 172.21.15.131:0/234682544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c240b7580 0x7f4c2415e9c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7f4c2b200640 1 -- 172.21.15.131:0/234682544 >> 172.21.15.131:0/234682544 conn(0x7f4c240d4d70 msgr2=0x7f4c240d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7f4c2b200640 1 -- 172.21.15.131:0/234682544 shutdown_connections 2024-04-28T12:37:55.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7f4c2b200640 1 -- 172.21.15.131:0/234682544 wait complete. 2024-04-28T12:37:55.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7fac3fe00640 1 -- 172.21.15.131:0/2829251935 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34037070 con 0x7fac400b50b0 2024-04-28T12:37:55.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7fac3fe00640 1 -- 172.21.15.131:0/2829251935 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac34032440 con 0x7fac400b50b0 2024-04-28T12:37:55.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.181+0000 7fac3fe00640 1 -- 172.21.15.131:0/2829251935 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34032810 con 0x7fac400b50b0 2024-04-28T12:37:55.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.182+0000 7fac46a00640 1 -- 172.21.15.131:0/2829251935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400b50b0 msgr2=0x7fac400b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.182+0000 7fac46a00640 1 --2- 172.21.15.131:0/2829251935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400b50b0 0x7fac400b5540 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fac4008d330 tx=0x7fac3402a160 comp rx=0 tx=0).stop 2024-04-28T12:37:55.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.182+0000 7fac46a00640 1 -- 172.21.15.131:0/2829251935 shutdown_connections 2024-04-28T12:37:55.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.182+0000 7fac46a00640 1 --2- 172.21.15.131:0/2829251935 >> v2:172.21.15.131:3300/0 conn(0x7fac400d1a00 0x7fac400b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.182+0000 7fac46a00640 1 --2- 172.21.15.131:0/2829251935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400b50b0 0x7fac400b5540 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.182+0000 7fac46a00640 1 --2- 172.21.15.131:0/2829251935 >> v2:172.21.15.157:3300/0 conn(0x7fac400d1620 0x7fac400d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.182+0000 7fac46a00640 1 -- 172.21.15.131:0/2829251935 >> 172.21.15.131:0/2829251935 conn(0x7fac400d4c30 msgr2=0x7fac400d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.183+0000 7f8a3f400640 1 -- 172.21.15.131:0/665906274 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a3801b070 con 0x7f8a4815ef00 2024-04-28T12:37:55.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.183+0000 7f8a4e800640 1 -- 172.21.15.131:0/665906274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a48069350 con 0x7f8a4815ef00 2024-04-28T12:37:55.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.184+0000 7f8a3f400640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8a2806c940 0x7f8a2806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.182+0000 7f54fb400640 1 -- 172.21.15.131:0/2974703319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54fc151e40 msgr2=0x7f54fc1fc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.184+0000 7f54fb400640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54fc151e40 0x7f54fc1fc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.184+0000 7f54fb400640 1 -- 172.21.15.131:0/2974703319 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc1fca30 con 0x7f54fc072000 2024-04-28T12:37:55.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.184+0000 7f5500a00640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54fc151e40 0x7f54fc1fc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:37:55.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.183+0000 7fb771e00640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb75406c9c0 0x7fb75406ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb764006fd0 tx=0x7fb764008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.184+0000 7f4c2b200640 1 Processor -- start 2024-04-28T12:37:55.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.184+0000 7f4c2b200640 1 -- start start 2024-04-28T12:37:55.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.183+0000 7fac46a00640 1 -- 172.21.15.131:0/2829251935 shutdown_connections 2024-04-28T12:37:55.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.183+0000 7fac46a00640 1 -- 172.21.15.131:0/2829251935 wait complete. 2024-04-28T12:37:55.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.185+0000 7fac46a00640 1 Processor -- start 2024-04-28T12:37:55.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.183+0000 7f9679e00640 1 Processor -- start 2024-04-28T12:37:55.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.183+0000 7f9679e00640 1 -- start start 2024-04-28T12:37:55.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.185+0000 7f9679e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f967414ceb0 0x7f96740b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.185+0000 7f54fb400640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc072000 0x7f54fc150380 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f54ec006fd0 tx=0x7f54ec00ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.185+0000 7f8a47400640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8a2806c940 0x7f8a2806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.185+0000 7fac46a00640 1 -- start start 2024-04-28T12:37:55.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.185+0000 7f9679e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f967414d290 con 0x7f96740cfe60 2024-04-28T12:37:55.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.185+0000 7f9679e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96740b5140 con 0x7f96740d1570 2024-04-28T12:37:55.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.185+0000 7f9679e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f96740b5420 con 0x7f967414ceb0 2024-04-28T12:37:55.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.186+0000 7f9672a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f967414ceb0 0x7f96740b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.186+0000 7fac46a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac400b50b0 0x7fac400cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.186+0000 7f9672a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f967414ceb0 0x7f96740b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42652/0 (socket says 172.21.15.131:42652) 2024-04-28T12:37:55.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.186+0000 7f9672a00640 1 -- 172.21.15.131:0/3518338795 learned_addr learned my addr 172.21.15.131:0/3518338795 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.186+0000 7f4c2b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c240b71a0 0x7f4c24161dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.186+0000 7fac46a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac400d1620 0x7fac40152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.186+0000 7fac46a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400d1a00 0x7fac400ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.186+0000 7fac46a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fac400b5a80 con 0x7fac400d1620 2024-04-28T12:37:55.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.186+0000 7fac46a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac400d1e60 con 0x7fac400b50b0 2024-04-28T12:37:55.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.186+0000 7fac46a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac400b5c00 con 0x7fac400d1a00 2024-04-28T12:37:55.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.186+0000 7f4c2b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c240b7580 0x7f4c24159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.186+0000 7f54f3e00640 1 -- 172.21.15.131:0/2974703319 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00f040 con 0x7f54fc072000 2024-04-28T12:37:55.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7fac45600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac400b50b0 0x7fac400cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f54f3e00640 1 -- 172.21.15.131:0/2974703319 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ec004030 con 0x7f54fc072000 2024-04-28T12:37:55.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f54f3e00640 1 -- 172.21.15.131:0/2974703319 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0043e0 con 0x7f54fc072000 2024-04-28T12:37:55.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f4c2b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2415ef00 0x7f4c2415be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f4c2b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4c240cfab0 con 0x7f4c240b71a0 2024-04-28T12:37:55.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f4c2b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4c240cf930 con 0x7f4c240b7580 2024-04-28T12:37:55.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f4c2b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4c240cfc30 con 0x7f4c2415ef00 2024-04-28T12:37:55.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7fac45600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac400b50b0 0x7fac400cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:42660/0 (socket says 172.21.15.131:42660) 2024-04-28T12:37:55.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7fac45600640 1 -- 172.21.15.131:0/2624042938 learned_addr learned my addr 172.21.15.131:0/2624042938 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7fac46000640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400d1a00 0x7fac400ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f4c23e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c240b71a0 0x7f4c24161dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f4c29200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2415ef00 0x7f4c2415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f4c29200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2415ef00 0x7f4c2415be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:46678/0 (socket says 172.21.15.131:46678) 2024-04-28T12:37:55.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f4c29200640 1 -- 172.21.15.131:0/3341687776 learned_addr learned my addr 172.21.15.131:0/3341687776 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f4c23e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c240b71a0 0x7f4c24161dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45348/0 (socket says 172.21.15.131:45348) 2024-04-28T12:37:55.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f5085200640 1 Processor -- start 2024-04-28T12:37:55.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.188+0000 7f4c23400640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c240b7580 0x7f4c24159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.188+0000 7f5085200640 1 -- start start 2024-04-28T12:37:55.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f5501400640 1 -- 172.21.15.131:0/2974703319 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54fc1fccd0 con 0x7f54fc072000 2024-04-28T12:37:55.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f5501400640 1 -- 172.21.15.131:0/2974703319 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f54fc1fd290 con 0x7f54fc072000 2024-04-28T12:37:55.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f9672a00640 1 -- 172.21.15.131:0/3518338795 >> v1:172.21.15.157:6789/0 conn(0x7f96740d1570 legacy=0x7f96740cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f9672a00640 1 -- 172.21.15.131:0/3518338795 >> v1:172.21.15.131:6789/0 conn(0x7f96740cfe60 legacy=0x7f967416d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.187+0000 7f9672a00640 1 -- 172.21.15.131:0/3518338795 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96740b5700 con 0x7f967414ceb0 2024-04-28T12:37:55.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.188+0000 7f8a47400640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8a2806c940 0x7f8a2806ee00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a40034000 tx=0x7f8a4002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.188+0000 7fac45600640 1 -- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400d1a00 msgr2=0x7fac400ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.188+0000 7fac45600640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400d1a00 0x7fac400ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.188+0000 7fac45600640 1 -- 172.21.15.131:0/2624042938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac400d1620 msgr2=0x7fac40152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.188+0000 7fac44c00640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac400d1620 0x7fac40152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7f4c23e00640 1 -- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2415ef00 msgr2=0x7f4c2415be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7f4c23e00640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2415ef00 0x7f4c2415be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7f4c23e00640 1 -- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c240b7580 msgr2=0x7f4c24159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7f4c23e00640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c240b7580 0x7f4c24159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7f4c23e00640 1 -- 172.21.15.131:0/3341687776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2415c960 con 0x7f4c240b71a0 2024-04-28T12:37:55.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7fac45600640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac400d1620 0x7fac40152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7fac45600640 1 -- 172.21.15.131:0/2624042938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac400cc2e0 con 0x7fac400b50b0 2024-04-28T12:37:55.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7f9672a00640 1 --2- 172.21.15.131:0/3518338795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f967414ceb0 0x7f96740b4a60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9664009870 tx=0x7f966402a310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d673f69d54dffa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7fac46000640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400d1a00 0x7fac400ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:37:55.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7fac44c00640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac400d1620 0x7fac40152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:55.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7f5501400640 1 -- 172.21.15.131:0/2974703319 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54fc069350 con 0x7f54fc072000 2024-04-28T12:37:55.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7f4c23e00640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c240b71a0 0x7f4c24161dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00c370 tx=0x7f4c0c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7f4756400640 1 Processor -- start 2024-04-28T12:37:55.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.189+0000 7fac45600640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac400b50b0 0x7fac400cbd00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fac3800ba80 tx=0x7fac3800bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.190+0000 7f9672000640 1 -- 172.21.15.131:0/3518338795 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664037070 con 0x7f967414ceb0 2024-04-28T12:37:55.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.190+0000 7f9672000640 1 -- 172.21.15.131:0/3518338795 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9664032640 con 0x7f967414ceb0 2024-04-28T12:37:55.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.190+0000 7f9672000640 1 -- 172.21.15.131:0/3518338795 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664037070 con 0x7f967414ceb0 2024-04-28T12:37:55.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.190+0000 7f5085200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5080166720 0x7f5080071900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f9679e00640 1 -- 172.21.15.131:0/3518338795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f967414ceb0 msgr2=0x7f96740b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f5085200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5080167bf0 con 0x7f5080071e40 2024-04-28T12:37:55.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f9679e00640 1 --2- 172.21.15.131:0/3518338795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f967414ceb0 0x7f96740b4a60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9664009870 tx=0x7f966402a310 comp rx=0 tx=0).stop 2024-04-28T12:37:55.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f5085200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5080166e30 con 0x7f50800b8640 2024-04-28T12:37:55.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f5085200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5080166b00 con 0x7f5080166720 2024-04-28T12:37:55.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f507f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5080166720 0x7f5080071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f507f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5080166720 0x7f5080071900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45366/0 (socket says 172.21.15.131:45366) 2024-04-28T12:37:55.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f507f400640 1 -- 172.21.15.131:0/3451636143 learned_addr learned my addr 172.21.15.131:0/3451636143 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f9679e00640 1 -- 172.21.15.131:0/3518338795 shutdown_connections 2024-04-28T12:37:55.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f9679e00640 1 --2- 172.21.15.131:0/3518338795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f967414ceb0 0x7f96740b4a60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f9679e00640 1 -- 172.21.15.131:0/3518338795 >> 172.21.15.131:0/3518338795 conn(0x7f96740b2900 msgr2=0x7f96740b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f8a3f400640 1 -- 172.21.15.131:0/665906274 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8a38056ba0 con 0x7f8a4815ef00 2024-04-28T12:37:55.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7fb76a000640 1 -- 172.21.15.131:0/1029735975 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb774161c40 con 0x7fb7740d3480 2024-04-28T12:37:55.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f4c1be00640 1 -- 172.21.15.131:0/3341687776 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c017070 con 0x7f4c240b71a0 2024-04-28T12:37:55.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.192+0000 7f4c2b200640 1 -- 172.21.15.131:0/3341687776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2415cc40 con 0x7f4c240b71a0 2024-04-28T12:37:55.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.192+0000 7f4c2b200640 1 -- 172.21.15.131:0/3341687776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c2415d200 con 0x7f4c240b71a0 2024-04-28T12:37:55.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.192+0000 7fac3d600640 1 -- 172.21.15.131:0/2624042938 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38019070 con 0x7fac400b50b0 2024-04-28T12:37:55.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.192+0000 7fac3d600640 1 -- 172.21.15.131:0/2624042938 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac3800cc90 con 0x7fac400b50b0 2024-04-28T12:37:55.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.192+0000 7fac3d600640 1 -- 172.21.15.131:0/2624042938 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38014680 con 0x7fac400b50b0 2024-04-28T12:37:55.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.193+0000 7f9679e00640 1 -- 172.21.15.131:0/3518338795 shutdown_connections 2024-04-28T12:37:55.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.193+0000 7f9679e00640 1 -- 172.21.15.131:0/3518338795 wait complete. 2024-04-28T12:37:55.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.193+0000 7fac46a00640 1 -- 172.21.15.131:0/2624042938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac400cc5c0 con 0x7fac400b50b0 2024-04-28T12:37:55.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.193+0000 7fac46a00640 1 -- 172.21.15.131:0/2624042938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac400ccb80 con 0x7fac400b50b0 2024-04-28T12:37:55.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.192+0000 7f507f400640 1 -- 172.21.15.131:0/3451636143 >> v1:172.21.15.157:6789/0 conn(0x7f50800b8640 legacy=0x7f50800d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.192+0000 7f507f400640 1 -- 172.21.15.131:0/3451636143 >> v1:172.21.15.131:6789/0 conn(0x7f5080071e40 legacy=0x7f50800b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:55.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.192+0000 7f507f400640 1 -- 172.21.15.131:0/3451636143 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5080167110 con 0x7f5080166720 2024-04-28T12:37:55.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.191+0000 7f4756400640 1 -- start start 2024-04-28T12:37:55.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.193+0000 7f4756400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47500b7580 0x7f475015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.193+0000 7f4756400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f475015ef00 0x7f47500cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.193+0000 7f4756400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47500c8830 con 0x7f47500b71a0 2024-04-28T12:37:55.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.193+0000 7f4756400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f47500d3cb0 con 0x7f47500b7580 2024-04-28T12:37:55.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.193+0000 7f4756400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f47500c8af0 con 0x7f475015ef00 2024-04-28T12:37:55.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.194+0000 7f474ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47500b7580 0x7f475015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.194+0000 7f474ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47500b7580 0x7f475015e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45382/0 (socket says 172.21.15.131:45382) 2024-04-28T12:37:55.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.194+0000 7f474ea00640 1 -- 172.21.15.131:0/2991456966 learned_addr learned my addr 172.21.15.131:0/2991456966 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.195+0000 7f474ea00640 1 -- 172.21.15.131:0/2991456966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f475015ef00 msgr2=0x7f47500cf930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-28T12:37:55.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.195+0000 7f474ea00640 1 --2- 172.21.15.131:0/2991456966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f475015ef00 0x7f47500cf930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.195+0000 7f4c2b200640 1 -- 172.21.15.131:0/3341687776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c24069350 con 0x7f4c240b71a0 2024-04-28T12:37:55.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.195+0000 7f474ea00640 1 -- 172.21.15.131:0/2991456966 >> v1:172.21.15.157:6789/0 conn(0x7f47500b71a0 legacy=0x7f47500d34d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-28T12:37:55.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.195+0000 7f474ea00640 1 -- 172.21.15.131:0/2991456966 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47500c8dd0 con 0x7f47500b7580 2024-04-28T12:37:55.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.195+0000 7f9679e00640 1 Processor -- start 2024-04-28T12:37:55.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.194+0000 7f54f3e00640 1 -- 172.21.15.131:0/2974703319 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54ec00c6d0 con 0x7f54fc072000 2024-04-28T12:37:55.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.195+0000 7f54f3e00640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f54dc06c8f0 0x7f54dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.195+0000 7f507f400640 1 --2- 172.21.15.131:0/3451636143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5080166720 0x7f5080071900 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5070009870 tx=0x7f507002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699671298166a70c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.196+0000 7f54f3e00640 1 -- 172.21.15.131:0/2974703319 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f54ec08c050 con 0x7f54fc072000 2024-04-28T12:37:55.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.196+0000 7f507e000640 1 -- 172.21.15.131:0/3451636143 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070037070 con 0x7f5080166720 2024-04-28T12:37:55.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.196+0000 7fac46a00640 1 -- 172.21.15.131:0/2624042938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac400ccd80 con 0x7fac400b50b0 2024-04-28T12:37:55.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.197+0000 7f507e000640 1 -- 172.21.15.131:0/3451636143 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5070032440 con 0x7f5080166720 2024-04-28T12:37:55.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.197+0000 7f507e000640 1 -- 172.21.15.131:0/3451636143 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070032810 con 0x7f5080166720 2024-04-28T12:37:55.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.197+0000 7fac3d600640 1 -- 172.21.15.131:0/2624042938 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac3801c070 con 0x7fac400b50b0 2024-04-28T12:37:55.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.196+0000 7f9679e00640 1 -- start start 2024-04-28T12:37:55.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.197+0000 7f9679e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96740cfe60 0x7f96740ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.197+0000 7f474ea00640 1 --2- 172.21.15.131:0/2991456966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47500b7580 0x7f475015e9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f473c009870 tx=0x7f473c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c99c603f5015b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.198+0000 7f5085200640 1 -- 172.21.15.131:0/3451636143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5080166720 msgr2=0x7f5080071900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.198+0000 7f5085200640 1 --2- 172.21.15.131:0/3451636143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5080166720 0x7f5080071900 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5070009870 tx=0x7f507002a180 comp rx=0 tx=0).stop 2024-04-28T12:37:55.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.197+0000 7f9679e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96740d1570 0x7f96740afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.197+0000 7f9679e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f967414ceb0 0x7f96741fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.197+0000 7f9679e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f96740d1a00 con 0x7f96740cfe60 2024-04-28T12:37:55.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.198+0000 7f5085200640 1 -- 172.21.15.131:0/3451636143 shutdown_connections 2024-04-28T12:37:55.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.198+0000 7f5085200640 1 --2- 172.21.15.131:0/3451636143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5080166720 0x7f5080071900 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.198+0000 7f5085200640 1 -- 172.21.15.131:0/3451636143 >> 172.21.15.131:0/3451636143 conn(0x7f50800d9170 msgr2=0x7f50800d9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.198+0000 7f54fbe00640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f54dc06c8f0 0x7f54dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.198+0000 7f5085200640 1 -- 172.21.15.131:0/3451636143 shutdown_connections 2024-04-28T12:37:55.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.197+0000 7f9679e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96740b5cc0 con 0x7f96740d1570 2024-04-28T12:37:55.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.197+0000 7f9679e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f967406f000 con 0x7f967414ceb0 2024-04-28T12:37:55.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.198+0000 7fac3d600640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac2006c9c0 0x7fac2006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.199+0000 7fac3d600640 1 -- 172.21.15.131:0/2624042938 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fac3808bc20 con 0x7fac400b50b0 2024-04-28T12:37:55.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.199+0000 7f4c1be00640 1 -- 172.21.15.131:0/3341687776 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c0c005ce0 con 0x7f4c240b71a0 2024-04-28T12:37:55.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.199+0000 7f4c1be00640 1 -- 172.21.15.131:0/3341687776 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c021060 con 0x7f4c240b71a0 2024-04-28T12:37:55.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.199+0000 7f8a3f400640 1 -- 172.21.15.131:0/665906274 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a38059200 con 0x7f8a4815ef00 2024-04-28T12:37:55.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.199+0000 7f5085200640 1 -- 172.21.15.131:0/3451636143 wait complete. 2024-04-28T12:37:55.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.200+0000 7fac44c00640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac2006c9c0 0x7fac2006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.200+0000 7f9673400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f967414ceb0 0x7f96741fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.200+0000 7f9673400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f967414ceb0 0x7f96741fc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:46696/0 (socket says 172.21.15.131:46696) 2024-04-28T12:37:55.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.200+0000 7f9673400640 1 -- 172.21.15.131:0/2643996339 learned_addr learned my addr 172.21.15.131:0/2643996339 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.200+0000 7f966be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96740d1570 0x7f96740afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.200+0000 7f9672a00640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96740cfe60 0x7f96740ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.199+0000 7f54fbe00640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f54dc06c8f0 0x7f54dc06edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f54f4007fd0 tx=0x7f54f4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.200+0000 7f4c1be00640 1 -- 172.21.15.131:0/3341687776 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c0c01a070 con 0x7f4c240b71a0 2024-04-28T12:37:55.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.201+0000 7fac44c00640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac2006c9c0 0x7fac2006ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fac4015ce80 tx=0x7fac2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.201+0000 7f9672a00640 1 -- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f967414ceb0 msgr2=0x7f96741fc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.201+0000 7f9672a00640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f967414ceb0 0x7f96741fc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.201+0000 7f5085200640 1 Processor -- start 2024-04-28T12:37:55.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.201+0000 7f9672a00640 1 -- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96740d1570 msgr2=0x7f96740afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.201+0000 7f9672a00640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96740d1570 0x7f96740afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.201+0000 7f9672a00640 1 -- 172.21.15.131:0/2643996339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9674165c80 con 0x7f96740cfe60 2024-04-28T12:37:55.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.201+0000 7f5085200640 1 -- start start 2024-04-28T12:37:55.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.201+0000 7f9673400640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f967414ceb0 0x7f96741fc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:37:55.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.202+0000 7f9672a00640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96740cfe60 0x7f96740ab480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9664031000 tx=0x7f96640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.202+0000 7f4c1be00640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4c0006c8f0 0x7f4c0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.202+0000 7f4c1be00640 1 -- 172.21.15.131:0/3341687776 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4c0c08b5f0 con 0x7f4c240b71a0 2024-04-28T12:37:55.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.203+0000 7f5085200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5080071e40 0x7f50801624d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.203+0000 7f5085200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50800b8640 0x7f50800bdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.203+0000 7f5085200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5080166720 0x7f5080164170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.203+0000 7f5085200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50800724c0 con 0x7f50800b8640 2024-04-28T12:37:55.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.203+0000 7f5085200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50800d3f00 con 0x7f5080166720 2024-04-28T12:37:55.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.203+0000 7f5085200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f50800d4080 con 0x7f5080071e40 2024-04-28T12:37:55.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.203+0000 7f4c23400640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4c0006c8f0 0x7f4c0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.203+0000 7f507ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50800b8640 0x7f50800bdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.203+0000 7f507ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50800b8640 0x7f50800bdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45404/0 (socket says 172.21.15.131:45404) 2024-04-28T12:37:55.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.203+0000 7f507ea00640 1 -- 172.21.15.131:0/3574960731 learned_addr learned my addr 172.21.15.131:0/3574960731 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.203+0000 7f507f400640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5080071e40 0x7f50801624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f507fe00640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5080166720 0x7f5080164170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f966b400640 1 -- 172.21.15.131:0/2643996339 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664037040 con 0x7f96740cfe60 2024-04-28T12:37:55.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f4c23400640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4c0006c8f0 0x7f4c0006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4c14034000 tx=0x7f4c1402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f4c1be00640 1 -- 172.21.15.131:0/3341687776 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c0c0591c0 con 0x7f4c240b71a0 2024-04-28T12:37:55.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f507ea00640 1 -- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5080071e40 msgr2=0x7f50801624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f507ea00640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5080071e40 0x7f50801624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f507ea00640 1 -- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5080166720 msgr2=0x7f5080164170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f507ea00640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5080166720 0x7f5080164170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f507ea00640 1 -- 172.21.15.131:0/3574960731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50800ccee0 con 0x7f50800b8640 2024-04-28T12:37:55.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f966b400640 1 -- 172.21.15.131:0/2643996339 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9664042070 con 0x7f96740cfe60 2024-04-28T12:37:55.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f966b400640 1 -- 172.21.15.131:0/2643996339 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966402ae60 con 0x7f96740cfe60 2024-04-28T12:37:55.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f9679e00640 1 -- 172.21.15.131:0/2643996339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9674165f60 con 0x7f96740cfe60 2024-04-28T12:37:55.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.205+0000 7f9679e00640 1 -- 172.21.15.131:0/2643996339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9674166a40 con 0x7f96740cfe60 2024-04-28T12:37:55.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f474e000640 1 -- 172.21.15.131:0/2991456966 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c037070 con 0x7f47500b7580 2024-04-28T12:37:55.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f474e000640 1 -- 172.21.15.131:0/2991456966 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f473c032440 con 0x7f47500b7580 2024-04-28T12:37:55.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.204+0000 7f474e000640 1 -- 172.21.15.131:0/2991456966 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c032810 con 0x7f47500b7580 2024-04-28T12:37:55.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.205+0000 7f507ea00640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50800b8640 0x7f50800bdb60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f506c00c430 tx=0x7f506c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.206+0000 7f5077400640 1 -- 172.21.15.131:0/3574960731 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c011790 con 0x7f50800b8640 2024-04-28T12:37:55.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.206+0000 7f5077400640 1 -- 172.21.15.131:0/3574960731 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506c017070 con 0x7f50800b8640 2024-04-28T12:37:55.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.206+0000 7f5077400640 1 -- 172.21.15.131:0/3574960731 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c011b70 con 0x7f50800b8640 2024-04-28T12:37:55.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.205+0000 7f4756400640 1 -- 172.21.15.131:0/2991456966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47500b7580 msgr2=0x7f475015e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.205+0000 7f4756400640 1 --2- 172.21.15.131:0/2991456966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47500b7580 0x7f475015e9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f473c009870 tx=0x7f473c02a1a0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.206+0000 7f4756400640 1 -- 172.21.15.131:0/2991456966 shutdown_connections 2024-04-28T12:37:55.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.206+0000 7f4756400640 1 --2- 172.21.15.131:0/2991456966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f475015ef00 0x7f47500cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.206+0000 7f4756400640 1 --2- 172.21.15.131:0/2991456966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47500b7580 0x7f475015e9c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.206+0000 7f4756400640 1 -- 172.21.15.131:0/2991456966 >> 172.21.15.131:0/2991456966 conn(0x7f47500d4d70 msgr2=0x7f47500d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.206+0000 7f4756400640 1 -- 172.21.15.131:0/2991456966 shutdown_connections 2024-04-28T12:37:55.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.206+0000 7f4756400640 1 -- 172.21.15.131:0/2991456966 wait complete. 2024-04-28T12:37:55.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.207+0000 7f9679e00640 1 -- 172.21.15.131:0/2643996339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9674069350 con 0x7f96740cfe60 2024-04-28T12:37:55.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.207+0000 7f5085200640 1 -- 172.21.15.131:0/3574960731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50800be3c0 con 0x7f50800b8640 2024-04-28T12:37:55.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.207+0000 7f5085200640 1 -- 172.21.15.131:0/3574960731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50800cd460 con 0x7f50800b8640 2024-04-28T12:37:55.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.208+0000 7f5085200640 1 -- 172.21.15.131:0/3574960731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5080069350 con 0x7f50800b8640 2024-04-28T12:37:55.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.209+0000 7f4756400640 1 Processor -- start 2024-04-28T12:37:55.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.209+0000 7f4756400640 1 -- start start 2024-04-28T12:37:55.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.209+0000 7f966b400640 1 -- 172.21.15.131:0/2643996339 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9664003a70 con 0x7f96740cfe60 2024-04-28T12:37:55.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.212+0000 7f54f3e00640 1 -- 172.21.15.131:0/2974703319 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f54ec017020 con 0x7f54fc072000 2024-04-28T12:37:55.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.211+0000 7f4756400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47500b71a0 0x7f475014b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.212+0000 7f5077400640 1 -- 172.21.15.131:0/3574960731 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f506c011d10 con 0x7f50800b8640 2024-04-28T12:37:55.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.211+0000 7f4756400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47500b7580 0x7f47500b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.211+0000 7f4756400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475015ef00 0x7f47500ba070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.211+0000 7f4756400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f475015f510 con 0x7f475015ef00 2024-04-28T12:37:55.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.211+0000 7f4756400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f475015f390 con 0x7f47500b71a0 2024-04-28T12:37:55.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.211+0000 7f4756400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f475015f690 con 0x7f47500b7580 2024-04-28T12:37:55.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.213+0000 7f5077400640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f506006c940 0x7f506006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.212+0000 7f474ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47500b7580 0x7f47500b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.213+0000 7f5077400640 1 -- 172.21.15.131:0/3574960731 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f506c08bc10 con 0x7f50800b8640 2024-04-28T12:37:55.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.212+0000 7f474ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47500b7580 0x7f47500b9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:46706/0 (socket says 172.21.15.131:46706) 2024-04-28T12:37:55.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.212+0000 7f474ea00640 1 -- 172.21.15.131:0/2824688352 learned_addr learned my addr 172.21.15.131:0/2824688352 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.212+0000 7f474f400640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47500b71a0 0x7f475014b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.213+0000 7f474ea00640 1 -- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47500b71a0 msgr2=0x7f475014b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.213+0000 7f474ea00640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47500b71a0 0x7f475014b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.213+0000 7f474ea00640 1 -- 172.21.15.131:0/2824688352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475015ef00 msgr2=0x7f47500ba070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.213+0000 7f474ea00640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475015ef00 0x7f47500ba070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.213+0000 7f474ea00640 1 -- 172.21.15.131:0/2824688352 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47500ce580 con 0x7f47500b7580 2024-04-28T12:37:55.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.213+0000 7f474ea00640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47500b7580 0x7f47500b9b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f473c009870 tx=0x7f473c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.213+0000 7f507f400640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f506006c940 0x7f506006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.214+0000 7f4747400640 1 -- 172.21.15.131:0/2824688352 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c041070 con 0x7f47500b7580 2024-04-28T12:37:55.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.214+0000 7f4747400640 1 -- 172.21.15.131:0/2824688352 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f473c03c930 con 0x7f47500b7580 2024-04-28T12:37:55.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.214+0000 7f4747400640 1 -- 172.21.15.131:0/2824688352 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c037040 con 0x7f47500b7580 2024-04-28T12:37:55.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.214+0000 7f966b400640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f965006c940 0x7f965006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.214+0000 7f966b400640 1 -- 172.21.15.131:0/2643996339 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f96640ac8d0 con 0x7f96740cfe60 2024-04-28T12:37:55.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.215+0000 7f507f400640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f506006c940 0x7f506006ee00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5070034000 tx=0x7f507002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.215+0000 7f966be00640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f965006c940 0x7f965006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.215+0000 7f4756400640 1 -- 172.21.15.131:0/2824688352 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47500ce860 con 0x7f47500b7580 2024-04-28T12:37:55.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.215+0000 7f4756400640 1 -- 172.21.15.131:0/2824688352 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f475015fad0 con 0x7f47500b7580 2024-04-28T12:37:55.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.216+0000 7f966be00640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f965006c940 0x7f965006ee00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9674148730 tx=0x7f965c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.217+0000 7f4756400640 1 -- 172.21.15.131:0/2824688352 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4750069350 con 0x7f47500b7580 2024-04-28T12:37:55.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.219+0000 7f4747400640 1 -- 172.21.15.131:0/2824688352 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f473c044070 con 0x7f47500b7580 2024-04-28T12:37:55.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.220+0000 7fac3d600640 1 -- 172.21.15.131:0/2624042938 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fac400ccd80 con 0x7fac400b50b0 2024-04-28T12:37:55.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.220+0000 7f966b400640 1 -- 172.21.15.131:0/2643996339 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9664032610 con 0x7f96740cfe60 2024-04-28T12:37:55.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.220+0000 7f4747400640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f473006c940 0x7f473006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.221+0000 7f4747400640 1 -- 172.21.15.131:0/2824688352 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f473c078860 con 0x7f47500b7580 2024-04-28T12:37:55.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.222+0000 7f474f400640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f473006c940 0x7f473006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.224+0000 7f474f400640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f473006c940 0x7f473006ee00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4740006fd0 tx=0x7f47400090e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.230+0000 7f5077400640 1 -- 172.21.15.131:0/3574960731 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f506c059690 con 0x7f50800b8640 2024-04-28T12:37:55.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.233+0000 7f4747400640 1 -- 172.21.15.131:0/2824688352 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f473c07a7c0 con 0x7f47500b7580 2024-04-28T12:37:55.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.619+0000 7fb773200640 1 -- 172.21.15.131:0/1029735975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb774069350 con 0x7fb7740d3480 2024-04-28T12:37:55.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.620+0000 7fb76a000640 1 -- 172.21.15.131:0/1029735975 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb76c059610 con 0x7fb7740d3480 2024-04-28T12:37:55.618 INFO:teuthology.orchestra.run.smithi131.stdout:55834574858 2024-04-28T12:37:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.624+0000 7fb773200640 1 -- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb75406c9c0 msgr2=0x7fb75406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.624+0000 7fb773200640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb75406c9c0 0x7fb75406ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb764006fd0 tx=0x7fb764008040 comp rx=0 tx=0).stop 2024-04-28T12:37:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.624+0000 7fb773200640 1 -- 172.21.15.131:0/1029735975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7740d3480 msgr2=0x7fb774150f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.625+0000 7fb773200640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7740d3480 0x7fb774150f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00caa0 tx=0x7fb76c00cf60 comp rx=0 tx=0).stop 2024-04-28T12:37:55.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.625+0000 7fb773200640 1 -- 172.21.15.131:0/1029735975 shutdown_connections 2024-04-28T12:37:55.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.625+0000 7fb773200640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb75406c9c0 0x7fb75406ee80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.625+0000 7fb773200640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7740d3480 0x7fb774150f20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.626+0000 7fb773200640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7740d1d90 0x7fb7740ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.626+0000 7fb773200640 1 --2- 172.21.15.131:0/1029735975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7740b5ca0 0x7fb774155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.626+0000 7fb773200640 1 -- 172.21.15.131:0/1029735975 >> 172.21.15.131:0/1029735975 conn(0x7fb7740d4c50 msgr2=0x7fb7740d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.626+0000 7fb773200640 1 -- 172.21.15.131:0/1029735975 shutdown_connections 2024-04-28T12:37:55.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.626+0000 7fb773200640 1 -- 172.21.15.131:0/1029735975 wait complete. 2024-04-28T12:37:55.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.633+0000 7f8a4e800640 1 -- 172.21.15.131:0/665906274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8a480d0370 con 0x7f8a4815ef00 2024-04-28T12:37:55.632 INFO:teuthology.orchestra.run.smithi131.stdout:55834574857 2024-04-28T12:37:55.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.634+0000 7f8a3f400640 1 -- 172.21.15.131:0/665906274 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8a38058b00 con 0x7f8a4815ef00 2024-04-28T12:37:55.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.638+0000 7f8a4e800640 1 -- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8a2806c940 msgr2=0x7f8a2806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.638+0000 7f8a4e800640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8a2806c940 0x7f8a2806ee00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a40034000 tx=0x7f8a4002f000 comp rx=0 tx=0).stop 2024-04-28T12:37:55.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.638+0000 7f8a4e800640 1 -- 172.21.15.131:0/665906274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a4815ef00 msgr2=0x7f8a480cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.638+0000 7f8a4e800640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a4815ef00 0x7f8a480cce00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800c560 tx=0x7f8a3800ca20 comp rx=0 tx=0).stop 2024-04-28T12:37:55.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.639+0000 7f8a4e800640 1 -- 172.21.15.131:0/665906274 shutdown_connections 2024-04-28T12:37:55.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.639+0000 7f8a4e800640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8a2806c940 0x7f8a2806ee00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.639+0000 7f8a4e800640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a4815ef00 0x7f8a480cce00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.639+0000 7f8a4e800640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8a480b7580 0x7f8a480ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.639+0000 7f8a4e800640 1 --2- 172.21.15.131:0/665906274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8a480b71a0 0x7f8a481f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.639+0000 7f8a4e800640 1 -- 172.21.15.131:0/665906274 >> 172.21.15.131:0/665906274 conn(0x7f8a480d4d70 msgr2=0x7f8a480cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.639+0000 7f8a4e800640 1 -- 172.21.15.131:0/665906274 shutdown_connections 2024-04-28T12:37:55.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.639+0000 7f8a4e800640 1 -- 172.21.15.131:0/665906274 wait complete. 2024-04-28T12:37:55.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.644+0000 7f4c2b200640 1 -- 172.21.15.131:0/3341687776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4c2408ee00 con 0x7f4c240b71a0 2024-04-28T12:37:55.644 INFO:teuthology.orchestra.run.smithi131.stdout:55834574857 2024-04-28T12:37:55.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.645+0000 7f4c1be00640 1 -- 172.21.15.131:0/3341687776 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4c0c05d4c0 con 0x7f4c240b71a0 2024-04-28T12:37:55.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.649+0000 7f4c2b200640 1 -- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4c0006c8f0 msgr2=0x7f4c0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.649+0000 7f4c2b200640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4c0006c8f0 0x7f4c0006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4c14034000 tx=0x7f4c1402e000 comp rx=0 tx=0).stop 2024-04-28T12:37:55.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.649+0000 7f4c2b200640 1 -- 172.21.15.131:0/3341687776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c240b71a0 msgr2=0x7f4c24161dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.649+0000 7f4c2b200640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c240b71a0 0x7f4c24161dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00c370 tx=0x7f4c0c00c830 comp rx=0 tx=0).stop 2024-04-28T12:37:55.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.650+0000 7f4c2b200640 1 -- 172.21.15.131:0/3341687776 shutdown_connections 2024-04-28T12:37:55.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.650+0000 7f4c2b200640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4c0006c8f0 0x7f4c0006edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.650+0000 7f4c2b200640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4c2415ef00 0x7f4c2415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.650+0000 7f4c2b200640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c240b7580 0x7f4c24159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.650+0000 7f4c2b200640 1 --2- 172.21.15.131:0/3341687776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c240b71a0 0x7f4c24161dd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.650+0000 7f4c2b200640 1 -- 172.21.15.131:0/3341687776 >> 172.21.15.131:0/3341687776 conn(0x7f4c240d4d70 msgr2=0x7f4c24162590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.650+0000 7f4c2b200640 1 -- 172.21.15.131:0/3341687776 shutdown_connections 2024-04-28T12:37:55.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.650+0000 7f4c2b200640 1 -- 172.21.15.131:0/3341687776 wait complete. 2024-04-28T12:37:55.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.655+0000 7f5501400640 1 -- 172.21.15.131:0/2974703319 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f54fc0afa00 con 0x7f54fc072000 2024-04-28T12:37:55.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-04-28T12:37:55.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.656+0000 7f54f3e00640 1 -- 172.21.15.131:0/2974703319 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f54ec05a120 con 0x7f54fc072000 2024-04-28T12:37:55.653 DEBUG:teuthology.parallel:result is None 2024-04-28T12:37:55.653 INFO:teuthology.orchestra.run.smithi131.stdout:55834574859 2024-04-28T12:37:55.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.659+0000 7fac46a00640 1 -- 172.21.15.131:0/2624042938 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fac40069350 con 0x7fac400b50b0 2024-04-28T12:37:55.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.660+0000 7f5501400640 1 -- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f54dc06c8f0 msgr2=0x7f54dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.660+0000 7f5501400640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f54dc06c8f0 0x7f54dc06edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f54f4007fd0 tx=0x7f54f4009040 comp rx=0 tx=0).stop 2024-04-28T12:37:55.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.660+0000 7f5501400640 1 -- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc072000 msgr2=0x7f54fc150380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.660+0000 7f5501400640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc072000 0x7f54fc150380 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f54ec006fd0 tx=0x7f54ec00ec70 comp rx=0 tx=0).stop 2024-04-28T12:37:55.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.660+0000 7fac3d600640 1 -- 172.21.15.131:0/2624042938 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fac380596a0 con 0x7fac400b50b0 2024-04-28T12:37:55.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.660+0000 7f5501400640 1 -- 172.21.15.131:0/2974703319 shutdown_connections 2024-04-28T12:37:55.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.660+0000 7f5501400640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f54dc06c8f0 0x7f54dc06edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.660+0000 7f5501400640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54fc151e40 0x7f54fc1fc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.660+0000 7f5501400640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54fc072000 0x7f54fc150380 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f54ec006fd0 tx=0x7f54ec00ec70 comp rx=0 tx=0).stop 2024-04-28T12:37:55.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.660+0000 7f5501400640 1 --2- 172.21.15.131:0/2974703319 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54fc068bb0 0x7f54fc154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.660+0000 7f5501400640 1 -- 172.21.15.131:0/2974703319 >> 172.21.15.131:0/2974703319 conn(0x7f54fc068090 msgr2=0x7f54fc06dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.658 INFO:teuthology.orchestra.run.smithi131.stdout:55834574857 2024-04-28T12:37:55.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.662+0000 7f5501400640 1 -- 172.21.15.131:0/2974703319 shutdown_connections 2024-04-28T12:37:55.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.662+0000 7f5501400640 1 -- 172.21.15.131:0/2974703319 wait complete. 2024-04-28T12:37:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.663+0000 7fac46a00640 1 -- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac2006c9c0 msgr2=0x7fac2006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.664+0000 7fac46a00640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac2006c9c0 0x7fac2006ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fac4015ce80 tx=0x7fac2c008040 comp rx=0 tx=0).stop 2024-04-28T12:37:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.664+0000 7fac46a00640 1 -- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac400b50b0 msgr2=0x7fac400cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.664+0000 7fac46a00640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac400b50b0 0x7fac400cbd00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fac3800ba80 tx=0x7fac3800bf40 comp rx=0 tx=0).stop 2024-04-28T12:37:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.664+0000 7fac46a00640 1 -- 172.21.15.131:0/2624042938 shutdown_connections 2024-04-28T12:37:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.664+0000 7fac46a00640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac2006c9c0 0x7fac2006ee80 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fac4015ce80 tx=0x7fac2c008040 comp rx=0 tx=0).stop 2024-04-28T12:37:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.664+0000 7fac46a00640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac400d1a00 0x7fac400ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.664+0000 7fac46a00640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac400d1620 0x7fac40152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.664+0000 7fac46a00640 1 --2- 172.21.15.131:0/2624042938 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac400b50b0 0x7fac400cbd00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.664+0000 7fac46a00640 1 -- 172.21.15.131:0/2624042938 >> 172.21.15.131:0/2624042938 conn(0x7fac400d4c30 msgr2=0x7fac400d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.664+0000 7fac46a00640 1 -- 172.21.15.131:0/2624042938 shutdown_connections 2024-04-28T12:37:55.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.664+0000 7f9679e00640 1 -- 172.21.15.131:0/2643996339 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9674071f50 con 0x7f96740cfe60 2024-04-28T12:37:55.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.664+0000 7fac46a00640 1 -- 172.21.15.131:0/2624042938 wait complete. 2024-04-28T12:37:55.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.665+0000 7f4756400640 1 -- 172.21.15.131:0/2824688352 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f475008ee00 con 0x7f47500b7580 2024-04-28T12:37:55.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.665+0000 7f966b400640 1 -- 172.21.15.131:0/2643996339 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f966407ab50 con 0x7f96740cfe60 2024-04-28T12:37:55.663 INFO:teuthology.orchestra.run.smithi131.stdout:55834574859 2024-04-28T12:37:55.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.666+0000 7f4747400640 1 -- 172.21.15.131:0/2824688352 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f473c07eac0 con 0x7f47500b7580 2024-04-28T12:37:55.664 INFO:teuthology.orchestra.run.smithi131.stdout:55834574858 2024-04-28T12:37:55.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2024-04-28T12:37:55.665 DEBUG:teuthology.parallel:result is None 2024-04-28T12:37:55.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.669+0000 7f9679e00640 1 -- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f965006c940 msgr2=0x7f965006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.669+0000 7f9679e00640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f965006c940 0x7f965006ee00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9674148730 tx=0x7f965c009040 comp rx=0 tx=0).stop 2024-04-28T12:37:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.669+0000 7f9679e00640 1 -- 172.21.15.131:0/2643996339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96740cfe60 msgr2=0x7f96740ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.669+0000 7f9679e00640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96740cfe60 0x7f96740ab480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9664031000 tx=0x7f96640046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.669+0000 7f9679e00640 1 -- 172.21.15.131:0/2643996339 shutdown_connections 2024-04-28T12:37:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.669+0000 7f9679e00640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f965006c940 0x7f965006ee00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.669+0000 7f9679e00640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f967414ceb0 0x7f96741fc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.669+0000 7f9679e00640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96740d1570 0x7f96740afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.669+0000 7f9679e00640 1 --2- 172.21.15.131:0/2643996339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96740cfe60 0x7f96740ab480 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.669+0000 7f9679e00640 1 -- 172.21.15.131:0/2643996339 >> 172.21.15.131:0/2643996339 conn(0x7f96740b2900 msgr2=0x7f96740b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.670+0000 7f9679e00640 1 -- 172.21.15.131:0/2643996339 shutdown_connections 2024-04-28T12:37:55.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.670+0000 7f9679e00640 1 -- 172.21.15.131:0/2643996339 wait complete. 2024-04-28T12:37:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.670+0000 7f4756400640 1 -- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f473006c940 msgr2=0x7f473006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.670+0000 7f4756400640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f473006c940 0x7f473006ee00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4740006fd0 tx=0x7f47400090e0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.670+0000 7f4756400640 1 -- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47500b7580 msgr2=0x7f47500b9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.670+0000 7f4756400640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47500b7580 0x7f47500b9b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f473c009870 tx=0x7f473c0046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.670+0000 7f4756400640 1 -- 172.21.15.131:0/2824688352 shutdown_connections 2024-04-28T12:37:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.670+0000 7f4756400640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f473006c940 0x7f473006ee00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.670+0000 7f4756400640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475015ef00 0x7f47500ba070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.670+0000 7f4756400640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47500b7580 0x7f47500b9b30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.670+0000 7f4756400640 1 --2- 172.21.15.131:0/2824688352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47500b71a0 0x7f475014b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.670+0000 7f4756400640 1 -- 172.21.15.131:0/2824688352 >> 172.21.15.131:0/2824688352 conn(0x7f47500d4d70 msgr2=0x7f47500d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.671+0000 7f4756400640 1 -- 172.21.15.131:0/2824688352 shutdown_connections 2024-04-28T12:37:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.671+0000 7f4756400640 1 -- 172.21.15.131:0/2824688352 wait complete. 2024-04-28T12:37:55.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.671+0000 7f5085200640 1 -- 172.21.15.131:0/3574960731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f50800bef00 con 0x7f50800b8640 2024-04-28T12:37:55.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.672+0000 7f5077400640 1 -- 172.21.15.131:0/3574960731 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f506c05d990 con 0x7f50800b8640 2024-04-28T12:37:55.669 INFO:teuthology.orchestra.run.smithi131.stdout:55834574859 2024-04-28T12:37:55.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.675+0000 7f5085200640 1 -- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f506006c940 msgr2=0x7f506006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.675+0000 7f5085200640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f506006c940 0x7f506006ee00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5070034000 tx=0x7f507002e000 comp rx=0 tx=0).stop 2024-04-28T12:37:55.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.675+0000 7f5085200640 1 -- 172.21.15.131:0/3574960731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50800b8640 msgr2=0x7f50800bdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.675+0000 7f5085200640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50800b8640 0x7f50800bdb60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f506c00c430 tx=0x7f506c00c8f0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.676+0000 7f5085200640 1 -- 172.21.15.131:0/3574960731 shutdown_connections 2024-04-28T12:37:55.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.676+0000 7f5085200640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f506006c940 0x7f506006ee00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.676+0000 7f5085200640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5080166720 0x7f5080164170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.676+0000 7f5085200640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50800b8640 0x7f50800bdb60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.676+0000 7f5085200640 1 --2- 172.21.15.131:0/3574960731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5080071e40 0x7f50801624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.676+0000 7f5085200640 1 -- 172.21.15.131:0/3574960731 >> 172.21.15.131:0/3574960731 conn(0x7f50800d9170 msgr2=0x7f508006f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.676+0000 7f5085200640 1 -- 172.21.15.131:0/3574960731 shutdown_connections 2024-04-28T12:37:55.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.676+0000 7f5085200640 1 -- 172.21.15.131:0/3574960731 wait complete. 2024-04-28T12:37:55.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2024-04-28T12:37:55.675 DEBUG:teuthology.parallel:result is None 2024-04-28T12:37:55.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2024-04-28T12:37:55.686 DEBUG:teuthology.parallel:result is None 2024-04-28T12:37:55.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-04-28T12:37:55.690 DEBUG:teuthology.parallel:result is None 2024-04-28T12:37:55.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-04-28T12:37:55.690 DEBUG:teuthology.parallel:result is None 2024-04-28T12:37:55.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-04-28T12:37:55.694 DEBUG:teuthology.parallel:result is None 2024-04-28T12:37:55.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-04-28T12:37:55.694 DEBUG:teuthology.parallel:result is None 2024-04-28T12:37:55.694 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-28T12:37:55.694 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:37:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.915+0000 7f769e600640 1 Processor -- start 2024-04-28T12:37:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.915+0000 7f769e600640 1 -- start start 2024-04-28T12:37:55.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.916+0000 7f769e600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f76980af710 0x7f76980afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.916+0000 7f769e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76980cf600 0x7f76980cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.916+0000 7f769e600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7698072db0 con 0x7f769815bb90 2024-04-28T12:37:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.916+0000 7f769e600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f76980cec10 con 0x7f76980af710 2024-04-28T12:37:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.916+0000 7f769e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f76980b00b0 con 0x7f76980cf600 2024-04-28T12:37:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.917+0000 7f7697e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f76980af710 0x7f76980afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.917+0000 7f7697e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f76980af710 0x7f76980afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45428/0 (socket says 172.21.15.131:45428) 2024-04-28T12:37:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.917+0000 7f7697e00640 1 -- 172.21.15.131:0/2489449010 learned_addr learned my addr 172.21.15.131:0/2489449010 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.917+0000 7f7697e00640 1 -- 172.21.15.131:0/2489449010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76980cf600 msgr2=0x7f76980cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.917+0000 7f7697e00640 1 --2- 172.21.15.131:0/2489449010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76980cf600 0x7f76980cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.917+0000 7f7697e00640 1 -- 172.21.15.131:0/2489449010 >> v1:172.21.15.157:6789/0 conn(0x7f769815bb90 legacy=0x7f76980737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.917+0000 7f7697e00640 1 -- 172.21.15.131:0/2489449010 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76980cee40 con 0x7f76980af710 2024-04-28T12:37:55.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.918+0000 7f7697e00640 1 --2- 172.21.15.131:0/2489449010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76980af710 0x7f76980afb70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f768400bb10 tx=0x7f768402c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75c415ed6219dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.918+0000 7f7697400640 1 -- 172.21.15.131:0/2489449010 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684032790 con 0x7f76980af710 2024-04-28T12:37:55.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.918+0000 7f7697400640 1 -- 172.21.15.131:0/2489449010 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7684038070 con 0x7f76980af710 2024-04-28T12:37:55.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.918+0000 7f7697400640 1 -- 172.21.15.131:0/2489449010 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684032b50 con 0x7f76980af710 2024-04-28T12:37:55.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.919+0000 7f769e600640 1 -- 172.21.15.131:0/2489449010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76980af710 msgr2=0x7f76980afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.919+0000 7f769e600640 1 --2- 172.21.15.131:0/2489449010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76980af710 0x7f76980afb70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f768400bb10 tx=0x7f768402c8d0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.919+0000 7f769e600640 1 -- 172.21.15.131:0/2489449010 shutdown_connections 2024-04-28T12:37:55.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.919+0000 7f769e600640 1 --2- 172.21.15.131:0/2489449010 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f76980cf600 0x7f76980cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.919+0000 7f769e600640 1 --2- 172.21.15.131:0/2489449010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76980af710 0x7f76980afb70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.919+0000 7f769e600640 1 -- 172.21.15.131:0/2489449010 >> 172.21.15.131:0/2489449010 conn(0x7f7698068090 msgr2=0x7f7698068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:55.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.919+0000 7f769e600640 1 -- 172.21.15.131:0/2489449010 shutdown_connections 2024-04-28T12:37:55.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.919+0000 7f769e600640 1 -- 172.21.15.131:0/2489449010 wait complete. 2024-04-28T12:37:55.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.921+0000 7f769e600640 1 Processor -- start 2024-04-28T12:37:55.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.921+0000 7f769e600640 1 -- start start 2024-04-28T12:37:55.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769e600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76980af710 0x7f76980b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76980cf600 0x7f7698152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f769815bb90 0x7f769815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769e600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7698073eb0 con 0x7f76980af710 2024-04-28T12:37:55.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f76980725a0 con 0x7f76980cf600 2024-04-28T12:37:55.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7698072ad0 con 0x7f769815bb90 2024-04-28T12:37:55.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76980af710 0x7f76980b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76980af710 0x7f76980b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45430/0 (socket says 172.21.15.131:45430) 2024-04-28T12:37:55.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769d200640 1 -- 172.21.15.131:0/296047634 learned_addr learned my addr 172.21.15.131:0/296047634 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:55.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f7697e00640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76980cf600 0x7f7698152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769d200640 1 -- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f769815bb90 msgr2=0x7f769815c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769dc00640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f769815bb90 0x7f769815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769d200640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f769815bb90 0x7f769815c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769d200640 1 -- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76980cf600 msgr2=0x7f7698152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:55.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769d200640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76980cf600 0x7f7698152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:55.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f769d200640 1 -- 172.21.15.131:0/296047634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76981511c0 con 0x7f76980af710 2024-04-28T12:37:55.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.922+0000 7f7697e00640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76980cf600 0x7f7698152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:55.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.923+0000 7f769d200640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76980af710 0x7f76980b07e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f768800ba50 tx=0x7f768800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.923+0000 7f7694c00640 1 -- 172.21.15.131:0/296047634 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688019070 con 0x7f76980af710 2024-04-28T12:37:55.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.923+0000 7f7694c00640 1 -- 172.21.15.131:0/296047634 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768800cc20 con 0x7f76980af710 2024-04-28T12:37:55.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.923+0000 7f7694c00640 1 -- 172.21.15.131:0/296047634 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688014680 con 0x7f76980af710 2024-04-28T12:37:55.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.923+0000 7f769e600640 1 -- 172.21.15.131:0/296047634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76981514a0 con 0x7f76980af710 2024-04-28T12:37:55.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.923+0000 7f769e600640 1 -- 172.21.15.131:0/296047634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7698151a60 con 0x7f76980af710 2024-04-28T12:37:55.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.924+0000 7f769e600640 1 -- 172.21.15.131:0/296047634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7698069350 con 0x7f76980af710 2024-04-28T12:37:55.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.925+0000 7f7694c00640 1 -- 172.21.15.131:0/296047634 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f768801c070 con 0x7f76980af710 2024-04-28T12:37:55.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.926+0000 7f7694c00640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f767806c8f0 0x7f767806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:55.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.926+0000 7f7694c00640 1 -- 172.21.15.131:0/296047634 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f768808bbc0 con 0x7f76980af710 2024-04-28T12:37:55.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.927+0000 7f7697e00640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f767806c8f0 0x7f767806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:55.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.928+0000 7f7697e00640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f767806c8f0 0x7f767806edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7684037000 tx=0x7f7684008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:55.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:55.930+0000 7f7694c00640 1 -- 172.21.15.131:0/296047634 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7688059650 con 0x7f76980af710 2024-04-28T12:37:56.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.112+0000 7f769e600640 1 -- 172.21.15.131:0/296047634 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76980b3d30 con 0x7f767806c8f0 2024-04-28T12:37:56.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.136+0000 7f7694c00640 1 -- 172.21.15.131:0/296047634 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f76980b3d30 con 0x7f767806c8f0 2024-04-28T12:37:56.137 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:56.139 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:37:56.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.143+0000 7f769e600640 1 -- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f767806c8f0 msgr2=0x7f767806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:56.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.143+0000 7f769e600640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f767806c8f0 0x7f767806edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7684037000 tx=0x7f7684008040 comp rx=0 tx=0).stop 2024-04-28T12:37:56.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.143+0000 7f769e600640 1 -- 172.21.15.131:0/296047634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76980af710 msgr2=0x7f76980b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:56.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.143+0000 7f769e600640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76980af710 0x7f76980b07e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f768800ba50 tx=0x7f768800bf10 comp rx=0 tx=0).stop 2024-04-28T12:37:56.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.143+0000 7f769e600640 1 -- 172.21.15.131:0/296047634 shutdown_connections 2024-04-28T12:37:56.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.143+0000 7f769e600640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f767806c8f0 0x7f767806edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.143+0000 7f769e600640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f769815bb90 0x7f769815c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.143+0000 7f769e600640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76980cf600 0x7f7698152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.143+0000 7f769e600640 1 --2- 172.21.15.131:0/296047634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76980af710 0x7f76980b07e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.143+0000 7f769e600640 1 -- 172.21.15.131:0/296047634 >> 172.21.15.131:0/296047634 conn(0x7f7698068090 msgr2=0x7f7698153c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:56.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.144+0000 7f769e600640 1 -- 172.21.15.131:0/296047634 shutdown_connections 2024-04-28T12:37:56.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.144+0000 7f769e600640 1 -- 172.21.15.131:0/296047634 wait complete. 2024-04-28T12:37:56.158 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-28T12:37:54.926171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231864,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742856,"statfs":{"total":773094113280,"available":772856684544,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005165"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980293+0000","last_change":"2024-04-28T12:37:21.812889+0000","last_active":"2024-04-28T12:37:27.980293+0000","last_peered":"2024-04-28T12:37:27.980293+0000","last_clean":"2024-04-28T12:37:27.980293+0000","last_became_active":"2024-04-28T12:37:21.812593+0000","last_became_peered":"2024-04-28T12:37:21.812593+0000","last_unstale":"2024-04-28T12:37:27.980293+0000","last_undegraded":"2024-04-28T12:37:27.980293+0000","last_fullsized":"2024-04-28T12:37:27.980293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:33.616030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978052+0000","last_change":"2024-04-28T12:37:22.794799+0000","last_active":"2024-04-28T12:37:27.978052+0000","last_peered":"2024-04-28T12:37:27.978052+0000","last_clean":"2024-04-28T12:37:27.978052+0000","last_became_active":"2024-04-28T12:37:22.794480+0000","last_became_peered":"2024-04-28T12:37:22.794480+0000","last_unstale":"2024-04-28T12:37:27.978052+0000","last_undegraded":"2024-04-28T12:37:27.978052+0000","last_fullsized":"2024-04-28T12:37:27.978052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:39.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976789+0000","last_change":"2024-04-28T12:37:26.883025+0000","last_active":"2024-04-28T12:37:27.976789+0000","last_peered":"2024-04-28T12:37:27.976789+0000","last_clean":"2024-04-28T12:37:27.976789+0000","last_became_active":"2024-04-28T12:37:26.882577+0000","last_became_peered":"2024-04-28T12:37:26.882577+0000","last_unstale":"2024-04-28T12:37:27.976789+0000","last_undegraded":"2024-04-28T12:37:27.976789+0000","last_fullsized":"2024-04-28T12:37:27.976789+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:09.634939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978274+0000","last_change":"2024-04-28T12:37:21.796325+0000","last_active":"2024-04-28T12:37:27.978274+0000","last_peered":"2024-04-28T12:37:27.978274+0000","last_clean":"2024-04-28T12:37:27.978274+0000","last_became_active":"2024-04-28T12:37:21.795878+0000","last_became_peered":"2024-04-28T12:37:21.795878+0000","last_unstale":"2024-04-28T12:37:27.978274+0000","last_undegraded":"2024-04-28T12:37:27.978274+0000","last_fullsized":"2024-04-28T12:37:27.978274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:55:28.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979406+0000","last_change":"2024-04-28T12:37:22.821287+0000","last_active":"2024-04-28T12:37:27.979406+0000","last_peered":"2024-04-28T12:37:27.979406+0000","last_clean":"2024-04-28T12:37:27.979406+0000","last_became_active":"2024-04-28T12:37:22.820999+0000","last_became_peered":"2024-04-28T12:37:22.820999+0000","last_unstale":"2024-04-28T12:37:27.979406+0000","last_undegraded":"2024-04-28T12:37:27.979406+0000","last_fullsized":"2024-04-28T12:37:27.979406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843680+0000","last_change":"2024-04-28T12:37:26.860456+0000","last_active":"2024-04-28T12:37:27.843680+0000","last_peered":"2024-04-28T12:37:27.843680+0000","last_clean":"2024-04-28T12:37:27.843680+0000","last_became_active":"2024-04-28T12:37:26.859360+0000","last_became_peered":"2024-04-28T12:37:26.859360+0000","last_unstale":"2024-04-28T12:37:27.843680+0000","last_undegraded":"2024-04-28T12:37:27.843680+0000","last_fullsized":"2024-04-28T12:37:27.843680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:17.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982655+0000","last_change":"2024-04-28T12:37:21.817405+0000","last_active":"2024-04-28T12:37:27.982655+0000","last_peered":"2024-04-28T12:37:27.982655+0000","last_clean":"2024-04-28T12:37:27.982655+0000","last_became_active":"2024-04-28T12:37:21.816935+0000","last_became_peered":"2024-04-28T12:37:21.816935+0000","last_unstale":"2024-04-28T12:37:27.982655+0000","last_undegraded":"2024-04-28T12:37:27.982655+0000","last_fullsized":"2024-04-28T12:37:27.982655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:32.253270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978974+0000","last_change":"2024-04-28T12:37:22.809893+0000","last_active":"2024-04-28T12:37:27.978974+0000","last_peered":"2024-04-28T12:37:27.978974+0000","last_clean":"2024-04-28T12:37:27.978974+0000","last_became_active":"2024-04-28T12:37:22.808912+0000","last_became_peered":"2024-04-28T12:37:22.808912+0000","last_unstale":"2024-04-28T12:37:27.978974+0000","last_undegraded":"2024-04-28T12:37:27.978974+0000","last_fullsized":"2024-04-28T12:37:27.978974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:39.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843302+0000","last_change":"2024-04-28T12:37:26.870945+0000","last_active":"2024-04-28T12:37:27.843302+0000","last_peered":"2024-04-28T12:37:27.843302+0000","last_clean":"2024-04-28T12:37:27.843302+0000","last_became_active":"2024-04-28T12:37:26.867901+0000","last_became_peered":"2024-04-28T12:37:26.867901+0000","last_unstale":"2024-04-28T12:37:27.843302+0000","last_undegraded":"2024-04-28T12:37:27.843302+0000","last_fullsized":"2024-04-28T12:37:27.843302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:50.835488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981355+0000","last_change":"2024-04-28T12:37:21.816630+0000","last_active":"2024-04-28T12:37:27.981355+0000","last_peered":"2024-04-28T12:37:27.981355+0000","last_clean":"2024-04-28T12:37:27.981355+0000","last_became_active":"2024-04-28T12:37:21.815735+0000","last_became_peered":"2024-04-28T12:37:21.815735+0000","last_unstale":"2024-04-28T12:37:27.981355+0000","last_undegraded":"2024-04-28T12:37:27.981355+0000","last_fullsized":"2024-04-28T12:37:27.981355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:46:26.715438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979607+0000","last_change":"2024-04-28T12:37:22.810382+0000","last_active":"2024-04-28T12:37:27.979607+0000","last_peered":"2024-04-28T12:37:27.979607+0000","last_clean":"2024-04-28T12:37:27.979607+0000","last_became_active":"2024-04-28T12:37:22.809175+0000","last_became_peered":"2024-04-28T12:37:22.809175+0000","last_unstale":"2024-04-28T12:37:27.979607+0000","last_undegraded":"2024-04-28T12:37:27.979607+0000","last_fullsized":"2024-04-28T12:37:27.979607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:45.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844410+0000","last_change":"2024-04-28T12:37:26.864046+0000","last_active":"2024-04-28T12:37:27.844410+0000","last_peered":"2024-04-28T12:37:27.844410+0000","last_clean":"2024-04-28T12:37:27.844410+0000","last_became_active":"2024-04-28T12:37:26.863585+0000","last_became_peered":"2024-04-28T12:37:26.863585+0000","last_unstale":"2024-04-28T12:37:27.844410+0000","last_undegraded":"2024-04-28T12:37:27.844410+0000","last_fullsized":"2024-04-28T12:37:27.844410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:26:03.360606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842825+0000","last_change":"2024-04-28T12:37:22.794854+0000","last_active":"2024-04-28T12:37:27.842825+0000","last_peered":"2024-04-28T12:37:27.842825+0000","last_clean":"2024-04-28T12:37:27.842825+0000","last_became_active":"2024-04-28T12:37:22.794522+0000","last_became_peered":"2024-04-28T12:37:22.794522+0000","last_unstale":"2024-04-28T12:37:27.842825+0000","last_undegraded":"2024-04-28T12:37:27.842825+0000","last_fullsized":"2024-04-28T12:37:27.842825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981814+0000","last_change":"2024-04-28T12:37:21.814514+0000","last_active":"2024-04-28T12:37:27.981814+0000","last_peered":"2024-04-28T12:37:27.981814+0000","last_clean":"2024-04-28T12:37:27.981814+0000","last_became_active":"2024-04-28T12:37:21.814226+0000","last_became_peered":"2024-04-28T12:37:21.814226+0000","last_unstale":"2024-04-28T12:37:27.981814+0000","last_undegraded":"2024-04-28T12:37:27.981814+0000","last_fullsized":"2024-04-28T12:37:27.981814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.846440+0000","last_change":"2024-04-28T12:37:26.869683+0000","last_active":"2024-04-28T12:37:27.846440+0000","last_peered":"2024-04-28T12:37:27.846440+0000","last_clean":"2024-04-28T12:37:27.846440+0000","last_became_active":"2024-04-28T12:37:26.869160+0000","last_became_peered":"2024-04-28T12:37:26.869160+0000","last_unstale":"2024-04-28T12:37:27.846440+0000","last_undegraded":"2024-04-28T12:37:27.846440+0000","last_fullsized":"2024-04-28T12:37:27.846440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977688+0000","last_change":"2024-04-28T12:37:22.797734+0000","last_active":"2024-04-28T12:37:27.977688+0000","last_peered":"2024-04-28T12:37:27.977688+0000","last_clean":"2024-04-28T12:37:27.977688+0000","last_became_active":"2024-04-28T12:37:22.797443+0000","last_became_peered":"2024-04-28T12:37:22.797443+0000","last_unstale":"2024-04-28T12:37:27.977688+0000","last_undegraded":"2024-04-28T12:37:27.977688+0000","last_fullsized":"2024-04-28T12:37:27.977688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978047+0000","last_change":"2024-04-28T12:37:21.798038+0000","last_active":"2024-04-28T12:37:27.978047+0000","last_peered":"2024-04-28T12:37:27.978047+0000","last_clean":"2024-04-28T12:37:27.978047+0000","last_became_active":"2024-04-28T12:37:21.797779+0000","last_became_peered":"2024-04-28T12:37:21.797779+0000","last_unstale":"2024-04-28T12:37:27.978047+0000","last_undegraded":"2024-04-28T12:37:27.978047+0000","last_fullsized":"2024-04-28T12:37:27.978047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:32.253270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845656+0000","last_change":"2024-04-28T12:37:26.869060+0000","last_active":"2024-04-28T12:37:27.845656+0000","last_peered":"2024-04-28T12:37:27.845656+0000","last_clean":"2024-04-28T12:37:27.845656+0000","last_became_active":"2024-04-28T12:37:26.867405+0000","last_became_peered":"2024-04-28T12:37:26.867405+0000","last_unstale":"2024-04-28T12:37:27.845656+0000","last_undegraded":"2024-04-28T12:37:27.845656+0000","last_fullsized":"2024-04-28T12:37:27.845656+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:29:29.617319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977155+0000","last_change":"2024-04-28T12:37:22.789718+0000","last_active":"2024-04-28T12:37:27.977155+0000","last_peered":"2024-04-28T12:37:27.977155+0000","last_clean":"2024-04-28T12:37:27.977155+0000","last_became_active":"2024-04-28T12:37:22.786954+0000","last_became_peered":"2024-04-28T12:37:22.786954+0000","last_unstale":"2024-04-28T12:37:27.977155+0000","last_undegraded":"2024-04-28T12:37:27.977155+0000","last_fullsized":"2024-04-28T12:37:27.977155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.975988+0000","last_change":"2024-04-28T12:37:21.804262+0000","last_active":"2024-04-28T12:37:27.975988+0000","last_peered":"2024-04-28T12:37:27.975988+0000","last_clean":"2024-04-28T12:37:27.975988+0000","last_became_active":"2024-04-28T12:37:21.803851+0000","last_became_peered":"2024-04-28T12:37:21.803851+0000","last_unstale":"2024-04-28T12:37:27.975988+0000","last_undegraded":"2024-04-28T12:37:27.975988+0000","last_fullsized":"2024-04-28T12:37:27.975988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:39.334819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978307+0000","last_change":"2024-04-28T12:37:26.884595+0000","last_active":"2024-04-28T12:37:27.978307+0000","last_peered":"2024-04-28T12:37:27.978307+0000","last_clean":"2024-04-28T12:37:27.978307+0000","last_became_active":"2024-04-28T12:37:26.883799+0000","last_became_peered":"2024-04-28T12:37:26.883799+0000","last_unstale":"2024-04-28T12:37:27.978307+0000","last_undegraded":"2024-04-28T12:37:27.978307+0000","last_fullsized":"2024-04-28T12:37:27.978307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:28.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844830+0000","last_change":"2024-04-28T12:37:22.778971+0000","last_active":"2024-04-28T12:37:27.844830+0000","last_peered":"2024-04-28T12:37:27.844830+0000","last_clean":"2024-04-28T12:37:27.844830+0000","last_became_active":"2024-04-28T12:37:22.778674+0000","last_became_peered":"2024-04-28T12:37:22.778674+0000","last_unstale":"2024-04-28T12:37:27.844830+0000","last_undegraded":"2024-04-28T12:37:27.844830+0000","last_fullsized":"2024-04-28T12:37:27.844830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:54.666754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979280+0000","last_change":"2024-04-28T12:37:21.816284+0000","last_active":"2024-04-28T12:37:27.979280+0000","last_peered":"2024-04-28T12:37:27.979280+0000","last_clean":"2024-04-28T12:37:27.979280+0000","last_became_active":"2024-04-28T12:37:21.815140+0000","last_became_peered":"2024-04-28T12:37:21.815140+0000","last_unstale":"2024-04-28T12:37:27.979280+0000","last_undegraded":"2024-04-28T12:37:27.979280+0000","last_fullsized":"2024-04-28T12:37:27.979280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:53.650061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.846043+0000","last_change":"2024-04-28T12:37:26.868136+0000","last_active":"2024-04-28T12:37:27.846043+0000","last_peered":"2024-04-28T12:37:27.846043+0000","last_clean":"2024-04-28T12:37:27.846043+0000","last_became_active":"2024-04-28T12:37:26.867663+0000","last_became_peered":"2024-04-28T12:37:26.867663+0000","last_unstale":"2024-04-28T12:37:27.846043+0000","last_undegraded":"2024-04-28T12:37:27.846043+0000","last_fullsized":"2024-04-28T12:37:27.846043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:45.428951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978823+0000","last_change":"2024-04-28T12:37:22.797968+0000","last_active":"2024-04-28T12:37:27.978823+0000","last_peered":"2024-04-28T12:37:27.978823+0000","last_clean":"2024-04-28T12:37:27.978823+0000","last_became_active":"2024-04-28T12:37:22.797258+0000","last_became_peered":"2024-04-28T12:37:22.797258+0000","last_unstale":"2024-04-28T12:37:27.978823+0000","last_undegraded":"2024-04-28T12:37:27.978823+0000","last_fullsized":"2024-04-28T12:37:27.978823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843883+0000","last_change":"2024-04-28T12:37:21.814135+0000","last_active":"2024-04-28T12:37:27.843883+0000","last_peered":"2024-04-28T12:37:27.843883+0000","last_clean":"2024-04-28T12:37:27.843883+0000","last_became_active":"2024-04-28T12:37:21.813800+0000","last_became_peered":"2024-04-28T12:37:21.813800+0000","last_unstale":"2024-04-28T12:37:27.843883+0000","last_undegraded":"2024-04-28T12:37:27.843883+0000","last_fullsized":"2024-04-28T12:37:27.843883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:49.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844709+0000","last_change":"2024-04-28T12:37:26.885109+0000","last_active":"2024-04-28T12:37:27.844709+0000","last_peered":"2024-04-28T12:37:27.844709+0000","last_clean":"2024-04-28T12:37:27.844709+0000","last_became_active":"2024-04-28T12:37:26.882922+0000","last_became_peered":"2024-04-28T12:37:26.882922+0000","last_unstale":"2024-04-28T12:37:27.844709+0000","last_undegraded":"2024-04-28T12:37:27.844709+0000","last_fullsized":"2024-04-28T12:37:27.844709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:20.147514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978601+0000","last_change":"2024-04-28T12:37:22.802331+0000","last_active":"2024-04-28T12:37:27.978601+0000","last_peered":"2024-04-28T12:37:27.978601+0000","last_clean":"2024-04-28T12:37:27.978601+0000","last_became_active":"2024-04-28T12:37:22.801939+0000","last_became_peered":"2024-04-28T12:37:22.801939+0000","last_unstale":"2024-04-28T12:37:27.978601+0000","last_undegraded":"2024-04-28T12:37:27.978601+0000","last_fullsized":"2024-04-28T12:37:27.978601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:59.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976553+0000","last_change":"2024-04-28T12:37:21.796110+0000","last_active":"2024-04-28T12:37:27.976553+0000","last_peered":"2024-04-28T12:37:27.976553+0000","last_clean":"2024-04-28T12:37:27.976553+0000","last_became_active":"2024-04-28T12:37:21.795405+0000","last_became_peered":"2024-04-28T12:37:21.795405+0000","last_unstale":"2024-04-28T12:37:27.976553+0000","last_undegraded":"2024-04-28T12:37:27.976553+0000","last_fullsized":"2024-04-28T12:37:27.976553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:53.650061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976613+0000","last_change":"2024-04-28T12:37:26.883160+0000","last_active":"2024-04-28T12:37:27.976613+0000","last_peered":"2024-04-28T12:37:27.976613+0000","last_clean":"2024-04-28T12:37:27.976613+0000","last_became_active":"2024-04-28T12:37:26.882684+0000","last_became_peered":"2024-04-28T12:37:26.882684+0000","last_unstale":"2024-04-28T12:37:27.976613+0000","last_undegraded":"2024-04-28T12:37:27.976613+0000","last_fullsized":"2024-04-28T12:37:27.976613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:19:01.590267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981129+0000","last_change":"2024-04-28T12:37:22.800581+0000","last_active":"2024-04-28T12:37:27.981129+0000","last_peered":"2024-04-28T12:37:27.981129+0000","last_clean":"2024-04-28T12:37:27.981129+0000","last_became_active":"2024-04-28T12:37:22.800241+0000","last_became_peered":"2024-04-28T12:37:22.800241+0000","last_unstale":"2024-04-28T12:37:27.981129+0000","last_undegraded":"2024-04-28T12:37:27.981129+0000","last_fullsized":"2024-04-28T12:37:27.981129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982900+0000","last_change":"2024-04-28T12:37:21.811668+0000","last_active":"2024-04-28T12:37:27.982900+0000","last_peered":"2024-04-28T12:37:27.982900+0000","last_clean":"2024-04-28T12:37:27.982900+0000","last_became_active":"2024-04-28T12:37:21.811372+0000","last_became_peered":"2024-04-28T12:37:21.811372+0000","last_unstale":"2024-04-28T12:37:27.982900+0000","last_undegraded":"2024-04-28T12:37:27.982900+0000","last_fullsized":"2024-04-28T12:37:27.982900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:00:04.591555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976863+0000","last_change":"2024-04-28T12:37:26.877753+0000","last_active":"2024-04-28T12:37:27.976863+0000","last_peered":"2024-04-28T12:37:27.976863+0000","last_clean":"2024-04-28T12:37:27.976863+0000","last_became_active":"2024-04-28T12:37:26.877456+0000","last_became_peered":"2024-04-28T12:37:26.877456+0000","last_unstale":"2024-04-28T12:37:27.976863+0000","last_undegraded":"2024-04-28T12:37:27.976863+0000","last_fullsized":"2024-04-28T12:37:27.976863+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:45.428951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979882+0000","last_change":"2024-04-28T12:37:22.801683+0000","last_active":"2024-04-28T12:37:27.979882+0000","last_peered":"2024-04-28T12:37:27.979882+0000","last_clean":"2024-04-28T12:37:27.979882+0000","last_became_active":"2024-04-28T12:37:22.801378+0000","last_became_peered":"2024-04-28T12:37:22.801378+0000","last_unstale":"2024-04-28T12:37:27.979882+0000","last_undegraded":"2024-04-28T12:37:27.979882+0000","last_fullsized":"2024-04-28T12:37:27.979882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:41.371396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981252+0000","last_change":"2024-04-28T12:37:21.791074+0000","last_active":"2024-04-28T12:37:27.981252+0000","last_peered":"2024-04-28T12:37:27.981252+0000","last_clean":"2024-04-28T12:37:27.981252+0000","last_became_active":"2024-04-28T12:37:21.790645+0000","last_became_peered":"2024-04-28T12:37:21.790645+0000","last_unstale":"2024-04-28T12:37:27.981252+0000","last_undegraded":"2024-04-28T12:37:27.981252+0000","last_fullsized":"2024-04-28T12:37:27.981252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977495+0000","last_change":"2024-04-28T12:37:26.873652+0000","last_active":"2024-04-28T12:37:27.977495+0000","last_peered":"2024-04-28T12:37:27.977495+0000","last_clean":"2024-04-28T12:37:27.977495+0000","last_became_active":"2024-04-28T12:37:26.873065+0000","last_became_peered":"2024-04-28T12:37:26.873065+0000","last_unstale":"2024-04-28T12:37:27.977495+0000","last_undegraded":"2024-04-28T12:37:27.977495+0000","last_fullsized":"2024-04-28T12:37:27.977495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845437+0000","last_change":"2024-04-28T12:37:22.794106+0000","last_active":"2024-04-28T12:37:27.845437+0000","last_peered":"2024-04-28T12:37:27.845437+0000","last_clean":"2024-04-28T12:37:27.845437+0000","last_became_active":"2024-04-28T12:37:22.789651+0000","last_became_peered":"2024-04-28T12:37:22.789651+0000","last_unstale":"2024-04-28T12:37:27.845437+0000","last_undegraded":"2024-04-28T12:37:27.845437+0000","last_fullsized":"2024-04-28T12:37:27.845437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:33.269962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979186+0000","last_change":"2024-04-28T12:37:21.810116+0000","last_active":"2024-04-28T12:37:27.979186+0000","last_peered":"2024-04-28T12:37:27.979186+0000","last_clean":"2024-04-28T12:37:27.979186+0000","last_became_active":"2024-04-28T12:37:21.809736+0000","last_became_peered":"2024-04-28T12:37:21.809736+0000","last_unstale":"2024-04-28T12:37:27.979186+0000","last_undegraded":"2024-04-28T12:37:27.979186+0000","last_fullsized":"2024-04-28T12:37:27.979186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:39.334819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981752+0000","last_change":"2024-04-28T12:37:26.867239+0000","last_active":"2024-04-28T12:37:27.981752+0000","last_peered":"2024-04-28T12:37:27.981752+0000","last_clean":"2024-04-28T12:37:27.981752+0000","last_became_active":"2024-04-28T12:37:26.866848+0000","last_became_peered":"2024-04-28T12:37:26.866848+0000","last_unstale":"2024-04-28T12:37:27.981752+0000","last_undegraded":"2024-04-28T12:37:27.981752+0000","last_fullsized":"2024-04-28T12:37:27.981752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:08:51.032543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976301+0000","last_change":"2024-04-28T12:37:22.796512+0000","last_active":"2024-04-28T12:37:27.976301+0000","last_peered":"2024-04-28T12:37:27.976301+0000","last_clean":"2024-04-28T12:37:27.976301+0000","last_became_active":"2024-04-28T12:37:22.793368+0000","last_became_peered":"2024-04-28T12:37:22.793368+0000","last_unstale":"2024-04-28T12:37:27.976301+0000","last_undegraded":"2024-04-28T12:37:27.976301+0000","last_fullsized":"2024-04-28T12:37:27.976301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.839137+0000","last_change":"2024-04-28T12:37:21.793644+0000","last_active":"2024-04-28T12:37:27.839137+0000","last_peered":"2024-04-28T12:37:27.839137+0000","last_clean":"2024-04-28T12:37:27.839137+0000","last_became_active":"2024-04-28T12:37:21.793358+0000","last_became_peered":"2024-04-28T12:37:21.793358+0000","last_unstale":"2024-04-28T12:37:27.839137+0000","last_undegraded":"2024-04-28T12:37:27.839137+0000","last_fullsized":"2024-04-28T12:37:27.839137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:55:28.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978474+0000","last_change":"2024-04-28T12:37:26.866571+0000","last_active":"2024-04-28T12:37:27.978474+0000","last_peered":"2024-04-28T12:37:27.978474+0000","last_clean":"2024-04-28T12:37:27.978474+0000","last_became_active":"2024-04-28T12:37:26.866042+0000","last_became_peered":"2024-04-28T12:37:26.866042+0000","last_unstale":"2024-04-28T12:37:27.978474+0000","last_undegraded":"2024-04-28T12:37:27.978474+0000","last_fullsized":"2024-04-28T12:37:27.978474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:04.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976223+0000","last_change":"2024-04-28T12:37:22.796747+0000","last_active":"2024-04-28T12:37:27.976223+0000","last_peered":"2024-04-28T12:37:27.976223+0000","last_clean":"2024-04-28T12:37:27.976223+0000","last_became_active":"2024-04-28T12:37:22.795677+0000","last_became_peered":"2024-04-28T12:37:22.795677+0000","last_unstale":"2024-04-28T12:37:27.976223+0000","last_undegraded":"2024-04-28T12:37:27.976223+0000","last_fullsized":"2024-04-28T12:37:27.976223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:05:18.644675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979595+0000","last_change":"2024-04-28T12:37:21.814725+0000","last_active":"2024-04-28T12:37:27.979595+0000","last_peered":"2024-04-28T12:37:27.979595+0000","last_clean":"2024-04-28T12:37:27.979595+0000","last_became_active":"2024-04-28T12:37:21.814516+0000","last_became_peered":"2024-04-28T12:37:21.814516+0000","last_unstale":"2024-04-28T12:37:27.979595+0000","last_undegraded":"2024-04-28T12:37:27.979595+0000","last_fullsized":"2024-04-28T12:37:27.979595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:49.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978718+0000","last_change":"2024-04-28T12:37:26.886450+0000","last_active":"2024-04-28T12:37:27.978718+0000","last_peered":"2024-04-28T12:37:27.978718+0000","last_clean":"2024-04-28T12:37:27.978718+0000","last_became_active":"2024-04-28T12:37:26.883398+0000","last_became_peered":"2024-04-28T12:37:26.883398+0000","last_unstale":"2024-04-28T12:37:27.978718+0000","last_undegraded":"2024-04-28T12:37:27.978718+0000","last_fullsized":"2024-04-28T12:37:27.978718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:04.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977534+0000","last_change":"2024-04-28T12:37:22.814159+0000","last_active":"2024-04-28T12:37:27.977534+0000","last_peered":"2024-04-28T12:37:27.977534+0000","last_clean":"2024-04-28T12:37:27.977534+0000","last_became_active":"2024-04-28T12:37:22.813889+0000","last_became_peered":"2024-04-28T12:37:22.813889+0000","last_unstale":"2024-04-28T12:37:27.977534+0000","last_undegraded":"2024-04-28T12:37:27.977534+0000","last_fullsized":"2024-04-28T12:37:27.977534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:16.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979063+0000","last_change":"2024-04-28T12:37:21.803169+0000","last_active":"2024-04-28T12:37:27.979063+0000","last_peered":"2024-04-28T12:37:27.979063+0000","last_clean":"2024-04-28T12:37:27.979063+0000","last_became_active":"2024-04-28T12:37:21.800812+0000","last_became_peered":"2024-04-28T12:37:21.800812+0000","last_unstale":"2024-04-28T12:37:27.979063+0000","last_undegraded":"2024-04-28T12:37:27.979063+0000","last_fullsized":"2024-04-28T12:37:27.979063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:55:28.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979702+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:37:27.979702+0000","last_peered":"2024-04-28T12:37:27.979702+0000","last_clean":"2024-04-28T12:37:27.979702+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:37:27.979702+0000","last_undegraded":"2024-04-28T12:37:27.979702+0000","last_fullsized":"2024-04-28T12:37:27.979702+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843882+0000","last_change":"2024-04-28T12:37:26.884731+0000","last_active":"2024-04-28T12:37:27.843882+0000","last_peered":"2024-04-28T12:37:27.843882+0000","last_clean":"2024-04-28T12:37:27.843882+0000","last_became_active":"2024-04-28T12:37:26.882799+0000","last_became_peered":"2024-04-28T12:37:26.882799+0000","last_unstale":"2024-04-28T12:37:27.843882+0000","last_undegraded":"2024-04-28T12:37:27.843882+0000","last_fullsized":"2024-04-28T12:37:27.843882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:51.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982421+0000","last_change":"2024-04-28T12:37:22.818975+0000","last_active":"2024-04-28T12:37:27.982421+0000","last_peered":"2024-04-28T12:37:27.982421+0000","last_clean":"2024-04-28T12:37:27.982421+0000","last_became_active":"2024-04-28T12:37:22.818677+0000","last_became_peered":"2024-04-28T12:37:22.818677+0000","last_unstale":"2024-04-28T12:37:27.982421+0000","last_undegraded":"2024-04-28T12:37:27.982421+0000","last_fullsized":"2024-04-28T12:37:27.982421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:41.371396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840221+0000","last_change":"2024-04-28T12:37:21.800771+0000","last_active":"2024-04-28T12:37:27.840221+0000","last_peered":"2024-04-28T12:37:27.840221+0000","last_clean":"2024-04-28T12:37:27.840221+0000","last_became_active":"2024-04-28T12:37:21.800451+0000","last_became_peered":"2024-04-28T12:37:21.800451+0000","last_unstale":"2024-04-28T12:37:27.840221+0000","last_undegraded":"2024-04-28T12:37:27.840221+0000","last_fullsized":"2024-04-28T12:37:27.840221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842666+0000","last_change":"2024-04-28T12:37:26.879284+0000","last_active":"2024-04-28T12:37:27.842666+0000","last_peered":"2024-04-28T12:37:27.842666+0000","last_clean":"2024-04-28T12:37:27.842666+0000","last_became_active":"2024-04-28T12:37:26.878864+0000","last_became_peered":"2024-04-28T12:37:26.878864+0000","last_unstale":"2024-04-28T12:37:27.842666+0000","last_undegraded":"2024-04-28T12:37:27.842666+0000","last_fullsized":"2024-04-28T12:37:27.842666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:45.428951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979686+0000","last_change":"2024-04-28T12:37:22.824563+0000","last_active":"2024-04-28T12:37:27.979686+0000","last_peered":"2024-04-28T12:37:27.979686+0000","last_clean":"2024-04-28T12:37:27.979686+0000","last_became_active":"2024-04-28T12:37:22.818493+0000","last_became_peered":"2024-04-28T12:37:22.818493+0000","last_unstale":"2024-04-28T12:37:27.979686+0000","last_undegraded":"2024-04-28T12:37:27.979686+0000","last_fullsized":"2024-04-28T12:37:27.979686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:00.502362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976419+0000","last_change":"2024-04-28T12:37:21.799906+0000","last_active":"2024-04-28T12:37:27.976419+0000","last_peered":"2024-04-28T12:37:27.976419+0000","last_clean":"2024-04-28T12:37:27.976419+0000","last_became_active":"2024-04-28T12:37:21.799470+0000","last_became_peered":"2024-04-28T12:37:21.799470+0000","last_unstale":"2024-04-28T12:37:27.976419+0000","last_undegraded":"2024-04-28T12:37:27.976419+0000","last_fullsized":"2024-04-28T12:37:27.976419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:49.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977214+0000","last_change":"2024-04-28T12:37:26.874939+0000","last_active":"2024-04-28T12:37:27.977214+0000","last_peered":"2024-04-28T12:37:27.977214+0000","last_clean":"2024-04-28T12:37:27.977214+0000","last_became_active":"2024-04-28T12:37:26.874472+0000","last_became_peered":"2024-04-28T12:37:26.874472+0000","last_unstale":"2024-04-28T12:37:27.977214+0000","last_undegraded":"2024-04-28T12:37:27.977214+0000","last_fullsized":"2024-04-28T12:37:27.977214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981950+0000","last_change":"2024-04-28T12:37:22.814584+0000","last_active":"2024-04-28T12:37:27.981950+0000","last_peered":"2024-04-28T12:37:27.981950+0000","last_clean":"2024-04-28T12:37:27.981950+0000","last_became_active":"2024-04-28T12:37:22.812752+0000","last_became_peered":"2024-04-28T12:37:22.812752+0000","last_unstale":"2024-04-28T12:37:27.981950+0000","last_undegraded":"2024-04-28T12:37:27.981950+0000","last_fullsized":"2024-04-28T12:37:27.981950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:48:32.880830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840416+0000","last_change":"2024-04-28T12:37:21.801322+0000","last_active":"2024-04-28T12:37:27.840416+0000","last_peered":"2024-04-28T12:37:27.840416+0000","last_clean":"2024-04-28T12:37:27.840416+0000","last_became_active":"2024-04-28T12:37:21.800437+0000","last_became_peered":"2024-04-28T12:37:21.800437+0000","last_unstale":"2024-04-28T12:37:27.840416+0000","last_undegraded":"2024-04-28T12:37:27.840416+0000","last_fullsized":"2024-04-28T12:37:27.840416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:33.616030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978163+0000","last_change":"2024-04-28T12:37:26.885854+0000","last_active":"2024-04-28T12:37:27.978163+0000","last_peered":"2024-04-28T12:37:27.978163+0000","last_clean":"2024-04-28T12:37:27.978163+0000","last_became_active":"2024-04-28T12:37:26.884298+0000","last_became_peered":"2024-04-28T12:37:26.884298+0000","last_unstale":"2024-04-28T12:37:27.978163+0000","last_undegraded":"2024-04-28T12:37:27.978163+0000","last_fullsized":"2024-04-28T12:37:27.978163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:19:01.590267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982107+0000","last_change":"2024-04-28T12:37:22.808698+0000","last_active":"2024-04-28T12:37:27.982107+0000","last_peered":"2024-04-28T12:37:27.982107+0000","last_clean":"2024-04-28T12:37:27.982107+0000","last_became_active":"2024-04-28T12:37:22.808363+0000","last_became_peered":"2024-04-28T12:37:22.808363+0000","last_unstale":"2024-04-28T12:37:27.982107+0000","last_undegraded":"2024-04-28T12:37:27.982107+0000","last_fullsized":"2024-04-28T12:37:27.982107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:47.618755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.846296+0000","last_change":"2024-04-28T12:37:21.794490+0000","last_active":"2024-04-28T12:37:27.846296+0000","last_peered":"2024-04-28T12:37:27.846296+0000","last_clean":"2024-04-28T12:37:27.846296+0000","last_became_active":"2024-04-28T12:37:21.794115+0000","last_became_peered":"2024-04-28T12:37:21.794115+0000","last_unstale":"2024-04-28T12:37:27.846296+0000","last_undegraded":"2024-04-28T12:37:27.846296+0000","last_fullsized":"2024-04-28T12:37:27.846296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:20:04.188478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844144+0000","last_change":"2024-04-28T12:37:26.867905+0000","last_active":"2024-04-28T12:37:27.844144+0000","last_peered":"2024-04-28T12:37:27.844144+0000","last_clean":"2024-04-28T12:37:27.844144+0000","last_became_active":"2024-04-28T12:37:26.867317+0000","last_became_peered":"2024-04-28T12:37:26.867317+0000","last_unstale":"2024-04-28T12:37:27.844144+0000","last_undegraded":"2024-04-28T12:37:27.844144+0000","last_fullsized":"2024-04-28T12:37:27.844144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:49.659072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980575+0000","last_change":"2024-04-28T12:37:22.802406+0000","last_active":"2024-04-28T12:37:27.980575+0000","last_peered":"2024-04-28T12:37:27.980575+0000","last_clean":"2024-04-28T12:37:27.980575+0000","last_became_active":"2024-04-28T12:37:22.802094+0000","last_became_peered":"2024-04-28T12:37:22.802094+0000","last_unstale":"2024-04-28T12:37:27.980575+0000","last_undegraded":"2024-04-28T12:37:27.980575+0000","last_fullsized":"2024-04-28T12:37:27.980575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:16.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.838661+0000","last_change":"2024-04-28T12:37:21.788140+0000","last_active":"2024-04-28T12:37:27.838661+0000","last_peered":"2024-04-28T12:37:27.838661+0000","last_clean":"2024-04-28T12:37:27.838661+0000","last_became_active":"2024-04-28T12:37:21.787727+0000","last_became_peered":"2024-04-28T12:37:21.787727+0000","last_unstale":"2024-04-28T12:37:27.838661+0000","last_undegraded":"2024-04-28T12:37:27.838661+0000","last_fullsized":"2024-04-28T12:37:27.838661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:29:20.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978548+0000","last_change":"2024-04-28T12:37:26.868358+0000","last_active":"2024-04-28T12:37:27.978548+0000","last_peered":"2024-04-28T12:37:27.978548+0000","last_clean":"2024-04-28T12:37:27.978548+0000","last_became_active":"2024-04-28T12:37:26.867972+0000","last_became_peered":"2024-04-28T12:37:26.867972+0000","last_unstale":"2024-04-28T12:37:27.978548+0000","last_undegraded":"2024-04-28T12:37:27.978548+0000","last_fullsized":"2024-04-28T12:37:27.978548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:28.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842120+0000","last_change":"2024-04-28T12:37:22.786192+0000","last_active":"2024-04-28T12:37:27.842120+0000","last_peered":"2024-04-28T12:37:27.842120+0000","last_clean":"2024-04-28T12:37:27.842120+0000","last_became_active":"2024-04-28T12:37:22.785730+0000","last_became_peered":"2024-04-28T12:37:22.785730+0000","last_unstale":"2024-04-28T12:37:27.842120+0000","last_undegraded":"2024-04-28T12:37:27.842120+0000","last_fullsized":"2024-04-28T12:37:27.842120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:42.893320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842909+0000","last_change":"2024-04-28T12:37:21.800159+0000","last_active":"2024-04-28T12:37:27.842909+0000","last_peered":"2024-04-28T12:37:27.842909+0000","last_clean":"2024-04-28T12:37:27.842909+0000","last_became_active":"2024-04-28T12:37:21.799817+0000","last_became_peered":"2024-04-28T12:37:21.799817+0000","last_unstale":"2024-04-28T12:37:27.842909+0000","last_undegraded":"2024-04-28T12:37:27.842909+0000","last_fullsized":"2024-04-28T12:37:27.842909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978507+0000","last_change":"2024-04-28T12:37:26.883264+0000","last_active":"2024-04-28T12:37:27.978507+0000","last_peered":"2024-04-28T12:37:27.978507+0000","last_clean":"2024-04-28T12:37:27.978507+0000","last_became_active":"2024-04-28T12:37:26.882787+0000","last_became_peered":"2024-04-28T12:37:26.882787+0000","last_unstale":"2024-04-28T12:37:27.978507+0000","last_undegraded":"2024-04-28T12:37:27.978507+0000","last_fullsized":"2024-04-28T12:37:27.978507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:11:24.219747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981395+0000","last_change":"2024-04-28T12:37:22.811865+0000","last_active":"2024-04-28T12:37:27.981395+0000","last_peered":"2024-04-28T12:37:27.981395+0000","last_clean":"2024-04-28T12:37:27.981395+0000","last_became_active":"2024-04-28T12:37:22.809479+0000","last_became_peered":"2024-04-28T12:37:22.809479+0000","last_unstale":"2024-04-28T12:37:27.981395+0000","last_undegraded":"2024-04-28T12:37:27.981395+0000","last_fullsized":"2024-04-28T12:37:27.981395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:39.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978873+0000","last_change":"2024-04-28T12:37:21.805620+0000","last_active":"2024-04-28T12:37:27.978873+0000","last_peered":"2024-04-28T12:37:27.978873+0000","last_clean":"2024-04-28T12:37:27.978873+0000","last_became_active":"2024-04-28T12:37:21.805397+0000","last_became_peered":"2024-04-28T12:37:21.805397+0000","last_unstale":"2024-04-28T12:37:27.978873+0000","last_undegraded":"2024-04-28T12:37:27.978873+0000","last_fullsized":"2024-04-28T12:37:27.978873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978474+0000","last_change":"2024-04-28T12:37:26.880544+0000","last_active":"2024-04-28T12:37:27.978474+0000","last_peered":"2024-04-28T12:37:27.978474+0000","last_clean":"2024-04-28T12:37:27.978474+0000","last_became_active":"2024-04-28T12:37:26.878960+0000","last_became_peered":"2024-04-28T12:37:26.878960+0000","last_unstale":"2024-04-28T12:37:27.978474+0000","last_undegraded":"2024-04-28T12:37:27.978474+0000","last_fullsized":"2024-04-28T12:37:27.978474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:28:21.376871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977145+0000","last_change":"2024-04-28T12:37:22.795607+0000","last_active":"2024-04-28T12:37:27.977145+0000","last_peered":"2024-04-28T12:37:27.977145+0000","last_clean":"2024-04-28T12:37:27.977145+0000","last_became_active":"2024-04-28T12:37:22.795262+0000","last_became_peered":"2024-04-28T12:37:22.795262+0000","last_unstale":"2024-04-28T12:37:27.977145+0000","last_undegraded":"2024-04-28T12:37:27.977145+0000","last_fullsized":"2024-04-28T12:37:27.977145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:42.893320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977438+0000","last_change":"2024-04-28T12:37:21.798231+0000","last_active":"2024-04-28T12:37:27.977438+0000","last_peered":"2024-04-28T12:37:27.977438+0000","last_clean":"2024-04-28T12:37:27.977438+0000","last_became_active":"2024-04-28T12:37:21.797978+0000","last_became_peered":"2024-04-28T12:37:21.797978+0000","last_unstale":"2024-04-28T12:37:27.977438+0000","last_undegraded":"2024-04-28T12:37:27.977438+0000","last_fullsized":"2024-04-28T12:37:27.977438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:20:04.188478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980535+0000","last_change":"2024-04-28T12:37:26.881273+0000","last_active":"2024-04-28T12:37:27.980535+0000","last_peered":"2024-04-28T12:37:27.980535+0000","last_clean":"2024-04-28T12:37:27.980535+0000","last_became_active":"2024-04-28T12:37:26.880845+0000","last_became_peered":"2024-04-28T12:37:26.880845+0000","last_unstale":"2024-04-28T12:37:27.980535+0000","last_undegraded":"2024-04-28T12:37:27.980535+0000","last_fullsized":"2024-04-28T12:37:27.980535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:09.634939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981232+0000","last_change":"2024-04-28T12:37:22.811901+0000","last_active":"2024-04-28T12:37:27.981232+0000","last_peered":"2024-04-28T12:37:27.981232+0000","last_clean":"2024-04-28T12:37:27.981232+0000","last_became_active":"2024-04-28T12:37:22.807998+0000","last_became_peered":"2024-04-28T12:37:22.807998+0000","last_unstale":"2024-04-28T12:37:27.981232+0000","last_undegraded":"2024-04-28T12:37:27.981232+0000","last_fullsized":"2024-04-28T12:37:27.981232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:29:25.559764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:41.723137+0000","last_change":"2024-04-28T12:37:21.802895+0000","last_active":"2024-04-28T12:37:41.723137+0000","last_peered":"2024-04-28T12:37:41.723137+0000","last_clean":"2024-04-28T12:37:41.723137+0000","last_became_active":"2024-04-28T12:37:21.801912+0000","last_became_peered":"2024-04-28T12:37:21.801912+0000","last_unstale":"2024-04-28T12:37:41.723137+0000","last_undegraded":"2024-04-28T12:37:41.723137+0000","last_fullsized":"2024-04-28T12:37:41.723137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:39.334819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977722+0000","last_change":"2024-04-28T12:37:26.873444+0000","last_active":"2024-04-28T12:37:27.977722+0000","last_peered":"2024-04-28T12:37:27.977722+0000","last_clean":"2024-04-28T12:37:27.977722+0000","last_became_active":"2024-04-28T12:37:26.872331+0000","last_became_peered":"2024-04-28T12:37:26.872331+0000","last_unstale":"2024-04-28T12:37:27.977722+0000","last_undegraded":"2024-04-28T12:37:27.977722+0000","last_fullsized":"2024-04-28T12:37:27.977722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:51.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845884+0000","last_change":"2024-04-28T12:37:22.786473+0000","last_active":"2024-04-28T12:37:27.845884+0000","last_peered":"2024-04-28T12:37:27.845884+0000","last_clean":"2024-04-28T12:37:27.845884+0000","last_became_active":"2024-04-28T12:37:22.785266+0000","last_became_peered":"2024-04-28T12:37:22.785266+0000","last_unstale":"2024-04-28T12:37:27.845884+0000","last_undegraded":"2024-04-28T12:37:27.845884+0000","last_fullsized":"2024-04-28T12:37:27.845884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:50.704472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976275+0000","last_change":"2024-04-28T12:37:21.810514+0000","last_active":"2024-04-28T12:37:27.976275+0000","last_peered":"2024-04-28T12:37:27.976275+0000","last_clean":"2024-04-28T12:37:27.976275+0000","last_became_active":"2024-04-28T12:37:21.810221+0000","last_became_peered":"2024-04-28T12:37:21.810221+0000","last_unstale":"2024-04-28T12:37:27.976275+0000","last_undegraded":"2024-04-28T12:37:27.976275+0000","last_fullsized":"2024-04-28T12:37:27.976275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845261+0000","last_change":"2024-04-28T12:37:26.871384+0000","last_active":"2024-04-28T12:37:27.845261+0000","last_peered":"2024-04-28T12:37:27.845261+0000","last_clean":"2024-04-28T12:37:27.845261+0000","last_became_active":"2024-04-28T12:37:26.868701+0000","last_became_peered":"2024-04-28T12:37:26.868701+0000","last_unstale":"2024-04-28T12:37:27.845261+0000","last_undegraded":"2024-04-28T12:37:27.845261+0000","last_fullsized":"2024-04-28T12:37:27.845261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:44.015884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979855+0000","last_change":"2024-04-28T12:37:22.821706+0000","last_active":"2024-04-28T12:37:27.979855+0000","last_peered":"2024-04-28T12:37:27.979855+0000","last_clean":"2024-04-28T12:37:27.979855+0000","last_became_active":"2024-04-28T12:37:22.821338+0000","last_became_peered":"2024-04-28T12:37:22.821338+0000","last_unstale":"2024-04-28T12:37:27.979855+0000","last_undegraded":"2024-04-28T12:37:27.979855+0000","last_fullsized":"2024-04-28T12:37:27.979855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980054+0000","last_change":"2024-04-28T12:37:21.803535+0000","last_active":"2024-04-28T12:37:27.980054+0000","last_peered":"2024-04-28T12:37:27.980054+0000","last_clean":"2024-04-28T12:37:27.980054+0000","last_became_active":"2024-04-28T12:37:21.801895+0000","last_became_peered":"2024-04-28T12:37:21.801895+0000","last_unstale":"2024-04-28T12:37:27.980054+0000","last_undegraded":"2024-04-28T12:37:27.980054+0000","last_fullsized":"2024-04-28T12:37:27.980054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:32.253270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976957+0000","last_change":"2024-04-28T12:37:26.861154+0000","last_active":"2024-04-28T12:37:27.976957+0000","last_peered":"2024-04-28T12:37:27.976957+0000","last_clean":"2024-04-28T12:37:27.976957+0000","last_became_active":"2024-04-28T12:37:26.860732+0000","last_became_peered":"2024-04-28T12:37:26.860732+0000","last_unstale":"2024-04-28T12:37:27.976957+0000","last_undegraded":"2024-04-28T12:37:27.976957+0000","last_fullsized":"2024-04-28T12:37:27.976957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:48:40.360635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979341+0000","last_change":"2024-04-28T12:37:22.817129+0000","last_active":"2024-04-28T12:37:27.979341+0000","last_peered":"2024-04-28T12:37:27.979341+0000","last_clean":"2024-04-28T12:37:27.979341+0000","last_became_active":"2024-04-28T12:37:22.816811+0000","last_became_peered":"2024-04-28T12:37:22.816811+0000","last_unstale":"2024-04-28T12:37:27.979341+0000","last_undegraded":"2024-04-28T12:37:27.979341+0000","last_fullsized":"2024-04-28T12:37:27.979341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:29:25.559764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843391+0000","last_change":"2024-04-28T12:37:21.805873+0000","last_active":"2024-04-28T12:37:27.843391+0000","last_peered":"2024-04-28T12:37:27.843391+0000","last_clean":"2024-04-28T12:37:27.843391+0000","last_became_active":"2024-04-28T12:37:21.805549+0000","last_became_peered":"2024-04-28T12:37:21.805549+0000","last_unstale":"2024-04-28T12:37:27.843391+0000","last_undegraded":"2024-04-28T12:37:27.843391+0000","last_fullsized":"2024-04-28T12:37:27.843391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976927+0000","last_change":"2024-04-28T12:37:26.883208+0000","last_active":"2024-04-28T12:37:27.976927+0000","last_peered":"2024-04-28T12:37:27.976927+0000","last_clean":"2024-04-28T12:37:27.976927+0000","last_became_active":"2024-04-28T12:37:26.882732+0000","last_became_peered":"2024-04-28T12:37:26.882732+0000","last_unstale":"2024-04-28T12:37:27.976927+0000","last_undegraded":"2024-04-28T12:37:27.976927+0000","last_fullsized":"2024-04-28T12:37:27.976927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:37:09.856024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978306+0000","last_change":"2024-04-28T12:37:26.888160+0000","last_active":"2024-04-28T12:37:27.978306+0000","last_peered":"2024-04-28T12:37:27.978306+0000","last_clean":"2024-04-28T12:37:27.978306+0000","last_became_active":"2024-04-28T12:37:26.886631+0000","last_became_peered":"2024-04-28T12:37:26.886631+0000","last_unstale":"2024-04-28T12:37:27.978306+0000","last_undegraded":"2024-04-28T12:37:27.978306+0000","last_fullsized":"2024-04-28T12:37:27.978306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:51.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.975748+0000","last_change":"2024-04-28T12:37:22.794465+0000","last_active":"2024-04-28T12:37:27.975748+0000","last_peered":"2024-04-28T12:37:27.975748+0000","last_clean":"2024-04-28T12:37:27.975748+0000","last_became_active":"2024-04-28T12:37:22.794138+0000","last_became_peered":"2024-04-28T12:37:22.794138+0000","last_unstale":"2024-04-28T12:37:27.975748+0000","last_undegraded":"2024-04-28T12:37:27.975748+0000","last_fullsized":"2024-04-28T12:37:27.975748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978354+0000","last_change":"2024-04-28T12:37:21.805697+0000","last_active":"2024-04-28T12:37:27.978354+0000","last_peered":"2024-04-28T12:37:27.978354+0000","last_clean":"2024-04-28T12:37:27.978354+0000","last_became_active":"2024-04-28T12:37:21.805449+0000","last_became_peered":"2024-04-28T12:37:21.805449+0000","last_unstale":"2024-04-28T12:37:27.978354+0000","last_undegraded":"2024-04-28T12:37:27.978354+0000","last_fullsized":"2024-04-28T12:37:27.978354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843785+0000","last_change":"2024-04-28T12:37:26.869931+0000","last_active":"2024-04-28T12:37:27.843785+0000","last_peered":"2024-04-28T12:37:27.843785+0000","last_clean":"2024-04-28T12:37:27.843785+0000","last_became_active":"2024-04-28T12:37:26.869441+0000","last_became_peered":"2024-04-28T12:37:26.869441+0000","last_unstale":"2024-04-28T12:37:27.843785+0000","last_undegraded":"2024-04-28T12:37:27.843785+0000","last_fullsized":"2024-04-28T12:37:27.843785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977035+0000","last_change":"2024-04-28T12:37:22.792490+0000","last_active":"2024-04-28T12:37:27.977035+0000","last_peered":"2024-04-28T12:37:27.977035+0000","last_clean":"2024-04-28T12:37:27.977035+0000","last_became_active":"2024-04-28T12:37:22.790799+0000","last_became_peered":"2024-04-28T12:37:22.790799+0000","last_unstale":"2024-04-28T12:37:27.977035+0000","last_undegraded":"2024-04-28T12:37:27.977035+0000","last_fullsized":"2024-04-28T12:37:27.977035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977471+0000","last_change":"2024-04-28T12:37:21.799595+0000","last_active":"2024-04-28T12:37:27.977471+0000","last_peered":"2024-04-28T12:37:27.977471+0000","last_clean":"2024-04-28T12:37:27.977471+0000","last_became_active":"2024-04-28T12:37:21.796500+0000","last_became_peered":"2024-04-28T12:37:21.796500+0000","last_unstale":"2024-04-28T12:37:27.977471+0000","last_undegraded":"2024-04-28T12:37:27.977471+0000","last_fullsized":"2024-04-28T12:37:27.977471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:33.616030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843252+0000","last_change":"2024-04-28T12:37:21.799592+0000","last_active":"2024-04-28T12:37:27.843252+0000","last_peered":"2024-04-28T12:37:27.843252+0000","last_clean":"2024-04-28T12:37:27.843252+0000","last_became_active":"2024-04-28T12:37:21.799085+0000","last_became_peered":"2024-04-28T12:37:21.799085+0000","last_unstale":"2024-04-28T12:37:27.843252+0000","last_undegraded":"2024-04-28T12:37:27.843252+0000","last_fullsized":"2024-04-28T12:37:27.843252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:53.650061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977307+0000","last_change":"2024-04-28T12:37:22.781856+0000","last_active":"2024-04-28T12:37:27.977307+0000","last_peered":"2024-04-28T12:37:27.977307+0000","last_clean":"2024-04-28T12:37:27.977307+0000","last_became_active":"2024-04-28T12:37:22.780347+0000","last_became_peered":"2024-04-28T12:37:22.780347+0000","last_unstale":"2024-04-28T12:37:27.977307+0000","last_undegraded":"2024-04-28T12:37:27.977307+0000","last_fullsized":"2024-04-28T12:37:27.977307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840176+0000","last_change":"2024-04-28T12:37:26.867761+0000","last_active":"2024-04-28T12:37:27.840176+0000","last_peered":"2024-04-28T12:37:27.840176+0000","last_clean":"2024-04-28T12:37:27.840176+0000","last_became_active":"2024-04-28T12:37:26.867277+0000","last_became_peered":"2024-04-28T12:37:26.867277+0000","last_unstale":"2024-04-28T12:37:27.840176+0000","last_undegraded":"2024-04-28T12:37:27.840176+0000","last_fullsized":"2024-04-28T12:37:27.840176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:12:40.786592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977843+0000","last_change":"2024-04-28T12:37:21.812666+0000","last_active":"2024-04-28T12:37:27.977843+0000","last_peered":"2024-04-28T12:37:27.977843+0000","last_clean":"2024-04-28T12:37:27.977843+0000","last_became_active":"2024-04-28T12:37:21.812384+0000","last_became_peered":"2024-04-28T12:37:21.812384+0000","last_unstale":"2024-04-28T12:37:27.977843+0000","last_undegraded":"2024-04-28T12:37:27.977843+0000","last_fullsized":"2024-04-28T12:37:27.977843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:29:20.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844555+0000","last_change":"2024-04-28T12:37:22.794127+0000","last_active":"2024-04-28T12:37:27.844555+0000","last_peered":"2024-04-28T12:37:27.844555+0000","last_clean":"2024-04-28T12:37:27.844555+0000","last_became_active":"2024-04-28T12:37:22.793770+0000","last_became_peered":"2024-04-28T12:37:22.793770+0000","last_unstale":"2024-04-28T12:37:27.844555+0000","last_undegraded":"2024-04-28T12:37:27.844555+0000","last_fullsized":"2024-04-28T12:37:27.844555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:00:05.608247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977959+0000","last_change":"2024-04-28T12:37:26.863918+0000","last_active":"2024-04-28T12:37:27.977959+0000","last_peered":"2024-04-28T12:37:27.977959+0000","last_clean":"2024-04-28T12:37:27.977959+0000","last_became_active":"2024-04-28T12:37:26.863558+0000","last_became_peered":"2024-04-28T12:37:26.863558+0000","last_unstale":"2024-04-28T12:37:27.977959+0000","last_undegraded":"2024-04-28T12:37:27.977959+0000","last_fullsized":"2024-04-28T12:37:27.977959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:49:38.590277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.841795+0000","last_change":"2024-04-28T12:37:22.786731+0000","last_active":"2024-04-28T12:37:27.841795+0000","last_peered":"2024-04-28T12:37:27.841795+0000","last_clean":"2024-04-28T12:37:27.841795+0000","last_became_active":"2024-04-28T12:37:22.786283+0000","last_became_peered":"2024-04-28T12:37:22.786283+0000","last_unstale":"2024-04-28T12:37:27.841795+0000","last_undegraded":"2024-04-28T12:37:27.841795+0000","last_fullsized":"2024-04-28T12:37:27.841795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.838968+0000","last_change":"2024-04-28T12:37:22.793953+0000","last_active":"2024-04-28T12:37:27.838968+0000","last_peered":"2024-04-28T12:37:27.838968+0000","last_clean":"2024-04-28T12:37:27.838968+0000","last_became_active":"2024-04-28T12:37:22.793683+0000","last_became_peered":"2024-04-28T12:37:22.793683+0000","last_unstale":"2024-04-28T12:37:27.838968+0000","last_undegraded":"2024-04-28T12:37:27.838968+0000","last_fullsized":"2024-04-28T12:37:27.838968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:33.269962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978735+0000","last_change":"2024-04-28T12:37:22.791966+0000","last_active":"2024-04-28T12:37:27.978735+0000","last_peered":"2024-04-28T12:37:27.978735+0000","last_clean":"2024-04-28T12:37:27.978735+0000","last_became_active":"2024-04-28T12:37:22.791432+0000","last_became_peered":"2024-04-28T12:37:22.791432+0000","last_unstale":"2024-04-28T12:37:27.978735+0000","last_undegraded":"2024-04-28T12:37:27.978735+0000","last_fullsized":"2024-04-28T12:37:27.978735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:47.693882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977566+0000","last_change":"2024-04-28T12:37:22.786169+0000","last_active":"2024-04-28T12:37:27.977566+0000","last_peered":"2024-04-28T12:37:27.977566+0000","last_clean":"2024-04-28T12:37:27.977566+0000","last_became_active":"2024-04-28T12:37:22.785899+0000","last_became_peered":"2024-04-28T12:37:22.785899+0000","last_unstale":"2024-04-28T12:37:27.977566+0000","last_undegraded":"2024-04-28T12:37:27.977566+0000","last_fullsized":"2024-04-28T12:37:27.977566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:06:12.328002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979102+0000","last_change":"2024-04-28T12:37:22.812114+0000","last_active":"2024-04-28T12:37:27.979102+0000","last_peered":"2024-04-28T12:37:27.979102+0000","last_clean":"2024-04-28T12:37:27.979102+0000","last_became_active":"2024-04-28T12:37:22.808394+0000","last_became_peered":"2024-04-28T12:37:22.808394+0000","last_unstale":"2024-04-28T12:37:27.979102+0000","last_undegraded":"2024-04-28T12:37:27.979102+0000","last_fullsized":"2024-04-28T12:37:27.979102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976044+0000","last_change":"2024-04-28T12:37:22.803864+0000","last_active":"2024-04-28T12:37:27.976044+0000","last_peered":"2024-04-28T12:37:27.976044+0000","last_clean":"2024-04-28T12:37:27.976044+0000","last_became_active":"2024-04-28T12:37:22.803568+0000","last_became_peered":"2024-04-28T12:37:22.803568+0000","last_unstale":"2024-04-28T12:37:27.976044+0000","last_undegraded":"2024-04-28T12:37:27.976044+0000","last_fullsized":"2024-04-28T12:37:27.976044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844461+0000","last_change":"2024-04-28T12:37:22.788814+0000","last_active":"2024-04-28T12:37:27.844461+0000","last_peered":"2024-04-28T12:37:27.844461+0000","last_clean":"2024-04-28T12:37:27.844461+0000","last_became_active":"2024-04-28T12:37:22.786716+0000","last_became_peered":"2024-04-28T12:37:22.786716+0000","last_unstale":"2024-04-28T12:37:27.844461+0000","last_undegraded":"2024-04-28T12:37:27.844461+0000","last_fullsized":"2024-04-28T12:37:27.844461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978543+0000","last_change":"2024-04-28T12:37:22.812645+0000","last_active":"2024-04-28T12:37:27.978543+0000","last_peered":"2024-04-28T12:37:27.978543+0000","last_clean":"2024-04-28T12:37:27.978543+0000","last_became_active":"2024-04-28T12:37:22.811034+0000","last_became_peered":"2024-04-28T12:37:22.811034+0000","last_unstale":"2024-04-28T12:37:27.978543+0000","last_undegraded":"2024-04-28T12:37:27.978543+0000","last_fullsized":"2024-04-28T12:37:27.978543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978281+0000","last_change":"2024-04-28T12:37:22.804060+0000","last_active":"2024-04-28T12:37:27.978281+0000","last_peered":"2024-04-28T12:37:27.978281+0000","last_clean":"2024-04-28T12:37:27.978281+0000","last_became_active":"2024-04-28T12:37:22.803757+0000","last_became_peered":"2024-04-28T12:37:22.803757+0000","last_unstale":"2024-04-28T12:37:27.978281+0000","last_undegraded":"2024-04-28T12:37:27.978281+0000","last_fullsized":"2024-04-28T12:37:27.978281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977634+0000","last_change":"2024-04-28T12:37:22.795301+0000","last_active":"2024-04-28T12:37:27.977634+0000","last_peered":"2024-04-28T12:37:27.977634+0000","last_clean":"2024-04-28T12:37:27.977634+0000","last_became_active":"2024-04-28T12:37:22.794917+0000","last_became_peered":"2024-04-28T12:37:22.794917+0000","last_unstale":"2024-04-28T12:37:27.977634+0000","last_undegraded":"2024-04-28T12:37:27.977634+0000","last_fullsized":"2024-04-28T12:37:27.977634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:45.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978597+0000","last_change":"2024-04-28T12:37:22.805760+0000","last_active":"2024-04-28T12:37:27.978597+0000","last_peered":"2024-04-28T12:37:27.978597+0000","last_clean":"2024-04-28T12:37:27.978597+0000","last_became_active":"2024-04-28T12:37:22.805238+0000","last_became_peered":"2024-04-28T12:37:22.805238+0000","last_unstale":"2024-04-28T12:37:27.978597+0000","last_undegraded":"2024-04-28T12:37:27.978597+0000","last_fullsized":"2024-04-28T12:37:27.978597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:16.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976930+0000","last_change":"2024-04-28T12:37:22.793207+0000","last_active":"2024-04-28T12:37:27.976930+0000","last_peered":"2024-04-28T12:37:27.976930+0000","last_clean":"2024-04-28T12:37:27.976930+0000","last_became_active":"2024-04-28T12:37:22.792816+0000","last_became_peered":"2024-04-28T12:37:22.792816+0000","last_unstale":"2024-04-28T12:37:27.976930+0000","last_undegraded":"2024-04-28T12:37:27.976930+0000","last_fullsized":"2024-04-28T12:37:27.976930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:55:36.416184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842925+0000","last_change":"2024-04-28T12:37:22.780772+0000","last_active":"2024-04-28T12:37:27.842925+0000","last_peered":"2024-04-28T12:37:27.842925+0000","last_clean":"2024-04-28T12:37:27.842925+0000","last_became_active":"2024-04-28T12:37:22.780190+0000","last_became_peered":"2024-04-28T12:37:22.780190+0000","last_unstale":"2024-04-28T12:37:27.842925+0000","last_undegraded":"2024-04-28T12:37:27.842925+0000","last_fullsized":"2024-04-28T12:37:27.842925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982761+0000","last_change":"2024-04-28T12:37:22.799792+0000","last_active":"2024-04-28T12:37:27.982761+0000","last_peered":"2024-04-28T12:37:27.982761+0000","last_clean":"2024-04-28T12:37:27.982761+0000","last_became_active":"2024-04-28T12:37:22.799472+0000","last_became_peered":"2024-04-28T12:37:22.799472+0000","last_unstale":"2024-04-28T12:37:27.982761+0000","last_undegraded":"2024-04-28T12:37:27.982761+0000","last_fullsized":"2024-04-28T12:37:27.982761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:47.693882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840300+0000","last_change":"2024-04-28T12:37:22.793748+0000","last_active":"2024-04-28T12:37:27.840300+0000","last_peered":"2024-04-28T12:37:27.840300+0000","last_clean":"2024-04-28T12:37:27.840300+0000","last_became_active":"2024-04-28T12:37:22.793461+0000","last_became_peered":"2024-04-28T12:37:22.793461+0000","last_unstale":"2024-04-28T12:37:27.840300+0000","last_undegraded":"2024-04-28T12:37:27.840300+0000","last_fullsized":"2024-04-28T12:37:27.840300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:00:05.608247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980132+0000","last_change":"2024-04-28T12:37:22.816550+0000","last_active":"2024-04-28T12:37:27.980132+0000","last_peered":"2024-04-28T12:37:27.980132+0000","last_clean":"2024-04-28T12:37:27.980132+0000","last_became_active":"2024-04-28T12:37:22.811895+0000","last_became_peered":"2024-04-28T12:37:22.811895+0000","last_unstale":"2024-04-28T12:37:27.980132+0000","last_undegraded":"2024-04-28T12:37:27.980132+0000","last_fullsized":"2024-04-28T12:37:27.980132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:24.827452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977952+0000","last_change":"2024-04-28T12:37:22.803909+0000","last_active":"2024-04-28T12:37:27.977952+0000","last_peered":"2024-04-28T12:37:27.977952+0000","last_clean":"2024-04-28T12:37:27.977952+0000","last_became_active":"2024-04-28T12:37:22.803577+0000","last_became_peered":"2024-04-28T12:37:22.803577+0000","last_unstale":"2024-04-28T12:37:27.977952+0000","last_undegraded":"2024-04-28T12:37:27.977952+0000","last_fullsized":"2024-04-28T12:37:27.977952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:59.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.975883+0000","last_change":"2024-04-28T12:37:22.808602+0000","last_active":"2024-04-28T12:37:27.975883+0000","last_peered":"2024-04-28T12:37:27.975883+0000","last_clean":"2024-04-28T12:37:27.975883+0000","last_became_active":"2024-04-28T12:37:22.808300+0000","last_became_peered":"2024-04-28T12:37:22.808300+0000","last_unstale":"2024-04-28T12:37:27.975883+0000","last_undegraded":"2024-04-28T12:37:27.975883+0000","last_fullsized":"2024-04-28T12:37:27.975883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977024+0000","last_change":"2024-04-28T12:37:22.792556+0000","last_active":"2024-04-28T12:37:27.977024+0000","last_peered":"2024-04-28T12:37:27.977024+0000","last_clean":"2024-04-28T12:37:27.977024+0000","last_became_active":"2024-04-28T12:37:22.789568+0000","last_became_peered":"2024-04-28T12:37:22.789568+0000","last_unstale":"2024-04-28T12:37:27.977024+0000","last_undegraded":"2024-04-28T12:37:27.977024+0000","last_fullsized":"2024-04-28T12:37:27.977024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:12:36.729037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977547+0000","last_change":"2024-04-28T12:37:22.798420+0000","last_active":"2024-04-28T12:37:27.977547+0000","last_peered":"2024-04-28T12:37:27.977547+0000","last_clean":"2024-04-28T12:37:27.977547+0000","last_became_active":"2024-04-28T12:37:22.798125+0000","last_became_peered":"2024-04-28T12:37:22.798125+0000","last_unstale":"2024-04-28T12:37:27.977547+0000","last_undegraded":"2024-04-28T12:37:27.977547+0000","last_fullsized":"2024-04-28T12:37:27.977547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:42.893320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845776+0000","last_change":"2024-04-28T12:37:22.799783+0000","last_active":"2024-04-28T12:37:27.845776+0000","last_peered":"2024-04-28T12:37:27.845776+0000","last_clean":"2024-04-28T12:37:27.845776+0000","last_became_active":"2024-04-28T12:37:22.796468+0000","last_became_peered":"2024-04-28T12:37:22.796468+0000","last_unstale":"2024-04-28T12:37:27.845776+0000","last_undegraded":"2024-04-28T12:37:27.845776+0000","last_fullsized":"2024-04-28T12:37:27.845776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:00.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844178+0000","last_change":"2024-04-28T12:37:22.806029+0000","last_active":"2024-04-28T12:37:27.844178+0000","last_peered":"2024-04-28T12:37:27.844178+0000","last_clean":"2024-04-28T12:37:27.844178+0000","last_became_active":"2024-04-28T12:37:22.802600+0000","last_became_peered":"2024-04-28T12:37:22.802600+0000","last_unstale":"2024-04-28T12:37:27.844178+0000","last_undegraded":"2024-04-28T12:37:27.844178+0000","last_fullsized":"2024-04-28T12:37:27.844178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844198+0000","last_change":"2024-04-28T12:37:22.821112+0000","last_active":"2024-04-28T12:37:27.844198+0000","last_peered":"2024-04-28T12:37:27.844198+0000","last_clean":"2024-04-28T12:37:27.844198+0000","last_became_active":"2024-04-28T12:37:22.820777+0000","last_became_peered":"2024-04-28T12:37:22.820777+0000","last_unstale":"2024-04-28T12:37:27.844198+0000","last_undegraded":"2024-04-28T12:37:27.844198+0000","last_fullsized":"2024-04-28T12:37:27.844198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981775+0000","last_change":"2024-04-28T12:37:22.825673+0000","last_active":"2024-04-28T12:37:27.981775+0000","last_peered":"2024-04-28T12:37:27.981775+0000","last_clean":"2024-04-28T12:37:27.981775+0000","last_became_active":"2024-04-28T12:37:22.820991+0000","last_became_peered":"2024-04-28T12:37:22.820991+0000","last_unstale":"2024-04-28T12:37:27.981775+0000","last_undegraded":"2024-04-28T12:37:27.981775+0000","last_fullsized":"2024-04-28T12:37:27.981775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:00.502362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976734+0000","last_change":"2024-04-28T12:37:22.814076+0000","last_active":"2024-04-28T12:37:27.976734+0000","last_peered":"2024-04-28T12:37:27.976734+0000","last_clean":"2024-04-28T12:37:27.976734+0000","last_became_active":"2024-04-28T12:37:22.813739+0000","last_became_peered":"2024-04-28T12:37:22.813739+0000","last_unstale":"2024-04-28T12:37:27.976734+0000","last_undegraded":"2024-04-28T12:37:27.976734+0000","last_fullsized":"2024-04-28T12:37:27.976734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980558+0000","last_change":"2024-04-28T12:37:22.799783+0000","last_active":"2024-04-28T12:37:27.980558+0000","last_peered":"2024-04-28T12:37:27.980558+0000","last_clean":"2024-04-28T12:37:27.980558+0000","last_became_active":"2024-04-28T12:37:22.799519+0000","last_became_peered":"2024-04-28T12:37:22.799519+0000","last_unstale":"2024-04-28T12:37:27.980558+0000","last_undegraded":"2024-04-28T12:37:27.980558+0000","last_fullsized":"2024-04-28T12:37:27.980558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:59.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978456+0000","last_change":"2024-04-28T12:37:22.805539+0000","last_active":"2024-04-28T12:37:27.978456+0000","last_peered":"2024-04-28T12:37:27.978456+0000","last_clean":"2024-04-28T12:37:27.978456+0000","last_became_active":"2024-04-28T12:37:22.804834+0000","last_became_peered":"2024-04-28T12:37:22.804834+0000","last_unstale":"2024-04-28T12:37:27.978456+0000","last_undegraded":"2024-04-28T12:37:27.978456+0000","last_fullsized":"2024-04-28T12:37:27.978456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:47.618755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981101+0000","last_change":"2024-04-28T12:37:22.800776+0000","last_active":"2024-04-28T12:37:27.981101+0000","last_peered":"2024-04-28T12:37:27.981101+0000","last_clean":"2024-04-28T12:37:27.981101+0000","last_became_active":"2024-04-28T12:37:22.798852+0000","last_became_peered":"2024-04-28T12:37:22.798852+0000","last_unstale":"2024-04-28T12:37:27.981101+0000","last_undegraded":"2024-04-28T12:37:27.981101+0000","last_fullsized":"2024-04-28T12:37:27.981101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:24.827452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978903+0000","last_change":"2024-04-28T12:37:22.819772+0000","last_active":"2024-04-28T12:37:27.978903+0000","last_peered":"2024-04-28T12:37:27.978903+0000","last_clean":"2024-04-28T12:37:27.978903+0000","last_became_active":"2024-04-28T12:37:22.813375+0000","last_became_peered":"2024-04-28T12:37:22.813375+0000","last_unstale":"2024-04-28T12:37:27.978903+0000","last_undegraded":"2024-04-28T12:37:27.978903+0000","last_fullsized":"2024-04-28T12:37:27.978903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978010+0000","last_change":"2024-04-28T12:37:22.784463+0000","last_active":"2024-04-28T12:37:27.978010+0000","last_peered":"2024-04-28T12:37:27.978010+0000","last_clean":"2024-04-28T12:37:27.978010+0000","last_became_active":"2024-04-28T12:37:22.784183+0000","last_became_peered":"2024-04-28T12:37:22.784183+0000","last_unstale":"2024-04-28T12:37:27.978010+0000","last_undegraded":"2024-04-28T12:37:27.978010+0000","last_fullsized":"2024-04-28T12:37:27.978010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976443+0000","last_change":"2024-04-28T12:37:22.802869+0000","last_active":"2024-04-28T12:37:27.976443+0000","last_peered":"2024-04-28T12:37:27.976443+0000","last_clean":"2024-04-28T12:37:27.976443+0000","last_became_active":"2024-04-28T12:37:22.802598+0000","last_became_peered":"2024-04-28T12:37:22.802598+0000","last_unstale":"2024-04-28T12:37:27.976443+0000","last_undegraded":"2024-04-28T12:37:27.976443+0000","last_fullsized":"2024-04-28T12:37:27.976443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981568+0000","last_change":"2024-04-28T12:37:22.797900+0000","last_active":"2024-04-28T12:37:27.981568+0000","last_peered":"2024-04-28T12:37:27.981568+0000","last_clean":"2024-04-28T12:37:27.981568+0000","last_became_active":"2024-04-28T12:37:22.797533+0000","last_became_peered":"2024-04-28T12:37:22.797533+0000","last_unstale":"2024-04-28T12:37:27.981568+0000","last_undegraded":"2024-04-28T12:37:27.981568+0000","last_fullsized":"2024-04-28T12:37:27.981568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:08:46.974988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:37:56.163 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:37:56.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.397+0000 7f7c4a200640 1 Processor -- start 2024-04-28T12:37:56.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.397+0000 7f7c4a200640 1 -- start start 2024-04-28T12:37:56.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.398+0000 7f7c4a200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7c440b71a0 0x7f7c440d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.398+0000 7f7c4a200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7c4415f0d0 0x7f7c440cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.398+0000 7f7c4a200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c440d3a10 con 0x7f7c440b7580 2024-04-28T12:37:56.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.398+0000 7f7c4a200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7c440c8830 con 0x7f7c4415f0d0 2024-04-28T12:37:56.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.398+0000 7f7c4a200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7c440d3cb0 con 0x7f7c440b71a0 2024-04-28T12:37:56.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.399+0000 7f7c43e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7c4415f0d0 0x7f7c440cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:56.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.399+0000 7f7c43e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7c4415f0d0 0x7f7c440cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45438/0 (socket says 172.21.15.131:45438) 2024-04-28T12:37:56.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.399+0000 7f7c43e00640 1 -- 172.21.15.131:0/580515550 learned_addr learned my addr 172.21.15.131:0/580515550 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:56.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.399+0000 7f7c43400640 1 --2- 172.21.15.131:0/580515550 >> v2:172.21.15.157:3300/0 conn(0x7f7c440b71a0 0x7f7c440d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:56.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.399+0000 7f7c43e00640 1 -- 172.21.15.131:0/580515550 >> v2:172.21.15.157:3300/0 conn(0x7f7c440b71a0 msgr2=0x7f7c440d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:56.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.399+0000 7f7c43e00640 1 --2- 172.21.15.131:0/580515550 >> v2:172.21.15.157:3300/0 conn(0x7f7c440b71a0 0x7f7c440d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.399+0000 7f7c43e00640 1 -- 172.21.15.131:0/580515550 >> v1:172.21.15.157:6789/0 conn(0x7f7c440b7580 legacy=0x7f7c4415e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:56.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.399+0000 7f7c43e00640 1 -- 172.21.15.131:0/580515550 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c440c8af0 con 0x7f7c4415f0d0 2024-04-28T12:37:56.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.401+0000 7f7c43e00640 1 --2- 172.21.15.131:0/580515550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c4415f0d0 0x7f7c440cf930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7c4408d350 tx=0x7f7c3c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61881c57d7daa414 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:56.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.401+0000 7f7c42000640 1 -- 172.21.15.131:0/580515550 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c036070 con 0x7f7c4415f0d0 2024-04-28T12:37:56.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.401+0000 7f7c42000640 1 -- 172.21.15.131:0/580515550 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c3c031600 con 0x7f7c4415f0d0 2024-04-28T12:37:56.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.401+0000 7f7c42000640 1 -- 172.21.15.131:0/580515550 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c0319d0 con 0x7f7c4415f0d0 2024-04-28T12:37:56.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.402+0000 7f7c4a200640 1 -- 172.21.15.131:0/580515550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c4415f0d0 msgr2=0x7f7c440cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:56.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.402+0000 7f7c4a200640 1 --2- 172.21.15.131:0/580515550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c4415f0d0 0x7f7c440cf930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7c4408d350 tx=0x7f7c3c02a6a0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.402+0000 7f7c4a200640 1 -- 172.21.15.131:0/580515550 shutdown_connections 2024-04-28T12:37:56.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.402+0000 7f7c4a200640 1 --2- 172.21.15.131:0/580515550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c4415f0d0 0x7f7c440cf930 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.402+0000 7f7c4a200640 1 --2- 172.21.15.131:0/580515550 >> v2:172.21.15.157:3300/0 conn(0x7f7c440b71a0 0x7f7c440d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.402+0000 7f7c4a200640 1 -- 172.21.15.131:0/580515550 >> 172.21.15.131:0/580515550 conn(0x7f7c440d4d70 msgr2=0x7f7c440d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:56.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.402+0000 7f7c4a200640 1 -- 172.21.15.131:0/580515550 shutdown_connections 2024-04-28T12:37:56.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.402+0000 7f7c4a200640 1 -- 172.21.15.131:0/580515550 wait complete. 2024-04-28T12:37:56.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.403+0000 7f7c4a200640 1 Processor -- start 2024-04-28T12:37:56.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.404+0000 7f7c4a200640 1 -- start start 2024-04-28T12:37:56.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.404+0000 7f7c4a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c440b71a0 0x7f7c44161d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.404+0000 7f7c4a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c440b7580 0x7f7c4415c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.404+0000 7f7c4a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c4415f0d0 0x7f7c4415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.404+0000 7f7c4a200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4415f500 con 0x7f7c440b71a0 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.404+0000 7f7c4a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c440c90b0 con 0x7f7c4415f0d0 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.405+0000 7f7c4a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4415f680 con 0x7f7c440b7580 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.405+0000 7f7c43400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c440b71a0 0x7f7c44161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.405+0000 7f7c43400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c440b71a0 0x7f7c44161d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45454/0 (socket says 172.21.15.131:45454) 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.405+0000 7f7c43400640 1 -- 172.21.15.131:0/4000541044 learned_addr learned my addr 172.21.15.131:0/4000541044 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.405+0000 7f7c43400640 1 -- 172.21.15.131:0/4000541044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c440b7580 msgr2=0x7f7c4415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.405+0000 7f7c43400640 1 --2- 172.21.15.131:0/4000541044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c440b7580 0x7f7c4415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.405+0000 7f7c43400640 1 -- 172.21.15.131:0/4000541044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c4415f0d0 msgr2=0x7f7c4415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.405+0000 7f7c43400640 1 --2- 172.21.15.131:0/4000541044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c4415f0d0 0x7f7c4415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.405+0000 7f7c43400640 1 -- 172.21.15.131:0/4000541044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c440cecf0 con 0x7f7c440b71a0 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.405+0000 7f7c43400640 1 --2- 172.21.15.131:0/4000541044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c440b71a0 0x7f7c44161d70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400e9b0 tx=0x7f7c3400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.406+0000 7f7c3b400640 1 -- 172.21.15.131:0/4000541044 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34017070 con 0x7f7c440b71a0 2024-04-28T12:37:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.406+0000 7f7c3b400640 1 -- 172.21.15.131:0/4000541044 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c3400c9a0 con 0x7f7c440b71a0 2024-04-28T12:37:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.406+0000 7f7c3b400640 1 -- 172.21.15.131:0/4000541044 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3400cd90 con 0x7f7c440b71a0 2024-04-28T12:37:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.406+0000 7f7c4a200640 1 -- 172.21.15.131:0/4000541044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c440cefd0 con 0x7f7c440b71a0 2024-04-28T12:37:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.406+0000 7f7c4a200640 1 -- 172.21.15.131:0/4000541044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c44161570 con 0x7f7c440b71a0 2024-04-28T12:37:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.407+0000 7f7c4a200640 1 -- 172.21.15.131:0/4000541044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c44069350 con 0x7f7c440b71a0 2024-04-28T12:37:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.408+0000 7f7c3b400640 1 -- 172.21.15.131:0/4000541044 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c3401a070 con 0x7f7c440b71a0 2024-04-28T12:37:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.409+0000 7f7c3b400640 1 --2- 172.21.15.131:0/4000541044 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7c2006c8f0 0x7f7c2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.409+0000 7f7c3b400640 1 -- 172.21.15.131:0/4000541044 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7c3408be60 con 0x7f7c440b71a0 2024-04-28T12:37:56.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.409+0000 7f7c42a00640 1 --2- 172.21.15.131:0/4000541044 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7c2006c8f0 0x7f7c2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:56.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.410+0000 7f7c42a00640 1 --2- 172.21.15.131:0/4000541044 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7c2006c8f0 0x7f7c2006edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c000f60 tx=0x7f7c2c0079d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:56.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.413+0000 7f7c3b400640 1 -- 172.21.15.131:0/4000541044 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7c34059f50 con 0x7f7c440b71a0 2024-04-28T12:37:56.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.593+0000 7f7c4a200640 1 -- 172.21.15.131:0/4000541044 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c4408ee00 con 0x7f7c2006c8f0 2024-04-28T12:37:56.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.617+0000 7f7c3b400640 1 -- 172.21.15.131:0/4000541044 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f7c4408ee00 con 0x7f7c2006c8f0 2024-04-28T12:37:56.616 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:56.620 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:37:56.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.624+0000 7f7c4a200640 1 -- 172.21.15.131:0/4000541044 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7c2006c8f0 msgr2=0x7f7c2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.624+0000 7f7c4a200640 1 --2- 172.21.15.131:0/4000541044 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7c2006c8f0 0x7f7c2006edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c000f60 tx=0x7f7c2c0079d0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.624+0000 7f7c4a200640 1 -- 172.21.15.131:0/4000541044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c440b71a0 msgr2=0x7f7c44161d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.624+0000 7f7c4a200640 1 --2- 172.21.15.131:0/4000541044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c440b71a0 0x7f7c44161d70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400e9b0 tx=0x7f7c3400ee70 comp rx=0 tx=0).stop 2024-04-28T12:37:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.624+0000 7f7c4a200640 1 -- 172.21.15.131:0/4000541044 shutdown_connections 2024-04-28T12:37:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.624+0000 7f7c4a200640 1 --2- 172.21.15.131:0/4000541044 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7c2006c8f0 0x7f7c2006edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.624+0000 7f7c4a200640 1 --2- 172.21.15.131:0/4000541044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c4415f0d0 0x7f7c4415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.624+0000 7f7c4a200640 1 --2- 172.21.15.131:0/4000541044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c440b7580 0x7f7c4415c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.624+0000 7f7c4a200640 1 --2- 172.21.15.131:0/4000541044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c440b71a0 0x7f7c44161d70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.624+0000 7f7c4a200640 1 -- 172.21.15.131:0/4000541044 >> 172.21.15.131:0/4000541044 conn(0x7f7c440d4d70 msgr2=0x7f7c440ba3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.624+0000 7f7c4a200640 1 -- 172.21.15.131:0/4000541044 shutdown_connections 2024-04-28T12:37:56.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.624+0000 7f7c4a200640 1 -- 172.21.15.131:0/4000541044 wait complete. 2024-04-28T12:37:56.641 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-28T12:37:54.926171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231864,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742856,"statfs":{"total":773094113280,"available":772856684544,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005165"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980293+0000","last_change":"2024-04-28T12:37:21.812889+0000","last_active":"2024-04-28T12:37:27.980293+0000","last_peered":"2024-04-28T12:37:27.980293+0000","last_clean":"2024-04-28T12:37:27.980293+0000","last_became_active":"2024-04-28T12:37:21.812593+0000","last_became_peered":"2024-04-28T12:37:21.812593+0000","last_unstale":"2024-04-28T12:37:27.980293+0000","last_undegraded":"2024-04-28T12:37:27.980293+0000","last_fullsized":"2024-04-28T12:37:27.980293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:33.616030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978052+0000","last_change":"2024-04-28T12:37:22.794799+0000","last_active":"2024-04-28T12:37:27.978052+0000","last_peered":"2024-04-28T12:37:27.978052+0000","last_clean":"2024-04-28T12:37:27.978052+0000","last_became_active":"2024-04-28T12:37:22.794480+0000","last_became_peered":"2024-04-28T12:37:22.794480+0000","last_unstale":"2024-04-28T12:37:27.978052+0000","last_undegraded":"2024-04-28T12:37:27.978052+0000","last_fullsized":"2024-04-28T12:37:27.978052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:39.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976789+0000","last_change":"2024-04-28T12:37:26.883025+0000","last_active":"2024-04-28T12:37:27.976789+0000","last_peered":"2024-04-28T12:37:27.976789+0000","last_clean":"2024-04-28T12:37:27.976789+0000","last_became_active":"2024-04-28T12:37:26.882577+0000","last_became_peered":"2024-04-28T12:37:26.882577+0000","last_unstale":"2024-04-28T12:37:27.976789+0000","last_undegraded":"2024-04-28T12:37:27.976789+0000","last_fullsized":"2024-04-28T12:37:27.976789+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:09.634939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978274+0000","last_change":"2024-04-28T12:37:21.796325+0000","last_active":"2024-04-28T12:37:27.978274+0000","last_peered":"2024-04-28T12:37:27.978274+0000","last_clean":"2024-04-28T12:37:27.978274+0000","last_became_active":"2024-04-28T12:37:21.795878+0000","last_became_peered":"2024-04-28T12:37:21.795878+0000","last_unstale":"2024-04-28T12:37:27.978274+0000","last_undegraded":"2024-04-28T12:37:27.978274+0000","last_fullsized":"2024-04-28T12:37:27.978274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:55:28.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979406+0000","last_change":"2024-04-28T12:37:22.821287+0000","last_active":"2024-04-28T12:37:27.979406+0000","last_peered":"2024-04-28T12:37:27.979406+0000","last_clean":"2024-04-28T12:37:27.979406+0000","last_became_active":"2024-04-28T12:37:22.820999+0000","last_became_peered":"2024-04-28T12:37:22.820999+0000","last_unstale":"2024-04-28T12:37:27.979406+0000","last_undegraded":"2024-04-28T12:37:27.979406+0000","last_fullsized":"2024-04-28T12:37:27.979406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843680+0000","last_change":"2024-04-28T12:37:26.860456+0000","last_active":"2024-04-28T12:37:27.843680+0000","last_peered":"2024-04-28T12:37:27.843680+0000","last_clean":"2024-04-28T12:37:27.843680+0000","last_became_active":"2024-04-28T12:37:26.859360+0000","last_became_peered":"2024-04-28T12:37:26.859360+0000","last_unstale":"2024-04-28T12:37:27.843680+0000","last_undegraded":"2024-04-28T12:37:27.843680+0000","last_fullsized":"2024-04-28T12:37:27.843680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:17.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982655+0000","last_change":"2024-04-28T12:37:21.817405+0000","last_active":"2024-04-28T12:37:27.982655+0000","last_peered":"2024-04-28T12:37:27.982655+0000","last_clean":"2024-04-28T12:37:27.982655+0000","last_became_active":"2024-04-28T12:37:21.816935+0000","last_became_peered":"2024-04-28T12:37:21.816935+0000","last_unstale":"2024-04-28T12:37:27.982655+0000","last_undegraded":"2024-04-28T12:37:27.982655+0000","last_fullsized":"2024-04-28T12:37:27.982655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:32.253270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978974+0000","last_change":"2024-04-28T12:37:22.809893+0000","last_active":"2024-04-28T12:37:27.978974+0000","last_peered":"2024-04-28T12:37:27.978974+0000","last_clean":"2024-04-28T12:37:27.978974+0000","last_became_active":"2024-04-28T12:37:22.808912+0000","last_became_peered":"2024-04-28T12:37:22.808912+0000","last_unstale":"2024-04-28T12:37:27.978974+0000","last_undegraded":"2024-04-28T12:37:27.978974+0000","last_fullsized":"2024-04-28T12:37:27.978974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:39.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843302+0000","last_change":"2024-04-28T12:37:26.870945+0000","last_active":"2024-04-28T12:37:27.843302+0000","last_peered":"2024-04-28T12:37:27.843302+0000","last_clean":"2024-04-28T12:37:27.843302+0000","last_became_active":"2024-04-28T12:37:26.867901+0000","last_became_peered":"2024-04-28T12:37:26.867901+0000","last_unstale":"2024-04-28T12:37:27.843302+0000","last_undegraded":"2024-04-28T12:37:27.843302+0000","last_fullsized":"2024-04-28T12:37:27.843302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:50.835488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981355+0000","last_change":"2024-04-28T12:37:21.816630+0000","last_active":"2024-04-28T12:37:27.981355+0000","last_peered":"2024-04-28T12:37:27.981355+0000","last_clean":"2024-04-28T12:37:27.981355+0000","last_became_active":"2024-04-28T12:37:21.815735+0000","last_became_peered":"2024-04-28T12:37:21.815735+0000","last_unstale":"2024-04-28T12:37:27.981355+0000","last_undegraded":"2024-04-28T12:37:27.981355+0000","last_fullsized":"2024-04-28T12:37:27.981355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:46:26.715438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979607+0000","last_change":"2024-04-28T12:37:22.810382+0000","last_active":"2024-04-28T12:37:27.979607+0000","last_peered":"2024-04-28T12:37:27.979607+0000","last_clean":"2024-04-28T12:37:27.979607+0000","last_became_active":"2024-04-28T12:37:22.809175+0000","last_became_peered":"2024-04-28T12:37:22.809175+0000","last_unstale":"2024-04-28T12:37:27.979607+0000","last_undegraded":"2024-04-28T12:37:27.979607+0000","last_fullsized":"2024-04-28T12:37:27.979607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:45.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844410+0000","last_change":"2024-04-28T12:37:26.864046+0000","last_active":"2024-04-28T12:37:27.844410+0000","last_peered":"2024-04-28T12:37:27.844410+0000","last_clean":"2024-04-28T12:37:27.844410+0000","last_became_active":"2024-04-28T12:37:26.863585+0000","last_became_peered":"2024-04-28T12:37:26.863585+0000","last_unstale":"2024-04-28T12:37:27.844410+0000","last_undegraded":"2024-04-28T12:37:27.844410+0000","last_fullsized":"2024-04-28T12:37:27.844410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:26:03.360606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842825+0000","last_change":"2024-04-28T12:37:22.794854+0000","last_active":"2024-04-28T12:37:27.842825+0000","last_peered":"2024-04-28T12:37:27.842825+0000","last_clean":"2024-04-28T12:37:27.842825+0000","last_became_active":"2024-04-28T12:37:22.794522+0000","last_became_peered":"2024-04-28T12:37:22.794522+0000","last_unstale":"2024-04-28T12:37:27.842825+0000","last_undegraded":"2024-04-28T12:37:27.842825+0000","last_fullsized":"2024-04-28T12:37:27.842825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981814+0000","last_change":"2024-04-28T12:37:21.814514+0000","last_active":"2024-04-28T12:37:27.981814+0000","last_peered":"2024-04-28T12:37:27.981814+0000","last_clean":"2024-04-28T12:37:27.981814+0000","last_became_active":"2024-04-28T12:37:21.814226+0000","last_became_peered":"2024-04-28T12:37:21.814226+0000","last_unstale":"2024-04-28T12:37:27.981814+0000","last_undegraded":"2024-04-28T12:37:27.981814+0000","last_fullsized":"2024-04-28T12:37:27.981814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.846440+0000","last_change":"2024-04-28T12:37:26.869683+0000","last_active":"2024-04-28T12:37:27.846440+0000","last_peered":"2024-04-28T12:37:27.846440+0000","last_clean":"2024-04-28T12:37:27.846440+0000","last_became_active":"2024-04-28T12:37:26.869160+0000","last_became_peered":"2024-04-28T12:37:26.869160+0000","last_unstale":"2024-04-28T12:37:27.846440+0000","last_undegraded":"2024-04-28T12:37:27.846440+0000","last_fullsized":"2024-04-28T12:37:27.846440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977688+0000","last_change":"2024-04-28T12:37:22.797734+0000","last_active":"2024-04-28T12:37:27.977688+0000","last_peered":"2024-04-28T12:37:27.977688+0000","last_clean":"2024-04-28T12:37:27.977688+0000","last_became_active":"2024-04-28T12:37:22.797443+0000","last_became_peered":"2024-04-28T12:37:22.797443+0000","last_unstale":"2024-04-28T12:37:27.977688+0000","last_undegraded":"2024-04-28T12:37:27.977688+0000","last_fullsized":"2024-04-28T12:37:27.977688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978047+0000","last_change":"2024-04-28T12:37:21.798038+0000","last_active":"2024-04-28T12:37:27.978047+0000","last_peered":"2024-04-28T12:37:27.978047+0000","last_clean":"2024-04-28T12:37:27.978047+0000","last_became_active":"2024-04-28T12:37:21.797779+0000","last_became_peered":"2024-04-28T12:37:21.797779+0000","last_unstale":"2024-04-28T12:37:27.978047+0000","last_undegraded":"2024-04-28T12:37:27.978047+0000","last_fullsized":"2024-04-28T12:37:27.978047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:32.253270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845656+0000","last_change":"2024-04-28T12:37:26.869060+0000","last_active":"2024-04-28T12:37:27.845656+0000","last_peered":"2024-04-28T12:37:27.845656+0000","last_clean":"2024-04-28T12:37:27.845656+0000","last_became_active":"2024-04-28T12:37:26.867405+0000","last_became_peered":"2024-04-28T12:37:26.867405+0000","last_unstale":"2024-04-28T12:37:27.845656+0000","last_undegraded":"2024-04-28T12:37:27.845656+0000","last_fullsized":"2024-04-28T12:37:27.845656+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:29:29.617319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977155+0000","last_change":"2024-04-28T12:37:22.789718+0000","last_active":"2024-04-28T12:37:27.977155+0000","last_peered":"2024-04-28T12:37:27.977155+0000","last_clean":"2024-04-28T12:37:27.977155+0000","last_became_active":"2024-04-28T12:37:22.786954+0000","last_became_peered":"2024-04-28T12:37:22.786954+0000","last_unstale":"2024-04-28T12:37:27.977155+0000","last_undegraded":"2024-04-28T12:37:27.977155+0000","last_fullsized":"2024-04-28T12:37:27.977155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.975988+0000","last_change":"2024-04-28T12:37:21.804262+0000","last_active":"2024-04-28T12:37:27.975988+0000","last_peered":"2024-04-28T12:37:27.975988+0000","last_clean":"2024-04-28T12:37:27.975988+0000","last_became_active":"2024-04-28T12:37:21.803851+0000","last_became_peered":"2024-04-28T12:37:21.803851+0000","last_unstale":"2024-04-28T12:37:27.975988+0000","last_undegraded":"2024-04-28T12:37:27.975988+0000","last_fullsized":"2024-04-28T12:37:27.975988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:39.334819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978307+0000","last_change":"2024-04-28T12:37:26.884595+0000","last_active":"2024-04-28T12:37:27.978307+0000","last_peered":"2024-04-28T12:37:27.978307+0000","last_clean":"2024-04-28T12:37:27.978307+0000","last_became_active":"2024-04-28T12:37:26.883799+0000","last_became_peered":"2024-04-28T12:37:26.883799+0000","last_unstale":"2024-04-28T12:37:27.978307+0000","last_undegraded":"2024-04-28T12:37:27.978307+0000","last_fullsized":"2024-04-28T12:37:27.978307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:28.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844830+0000","last_change":"2024-04-28T12:37:22.778971+0000","last_active":"2024-04-28T12:37:27.844830+0000","last_peered":"2024-04-28T12:37:27.844830+0000","last_clean":"2024-04-28T12:37:27.844830+0000","last_became_active":"2024-04-28T12:37:22.778674+0000","last_became_peered":"2024-04-28T12:37:22.778674+0000","last_unstale":"2024-04-28T12:37:27.844830+0000","last_undegraded":"2024-04-28T12:37:27.844830+0000","last_fullsized":"2024-04-28T12:37:27.844830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:54.666754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979280+0000","last_change":"2024-04-28T12:37:21.816284+0000","last_active":"2024-04-28T12:37:27.979280+0000","last_peered":"2024-04-28T12:37:27.979280+0000","last_clean":"2024-04-28T12:37:27.979280+0000","last_became_active":"2024-04-28T12:37:21.815140+0000","last_became_peered":"2024-04-28T12:37:21.815140+0000","last_unstale":"2024-04-28T12:37:27.979280+0000","last_undegraded":"2024-04-28T12:37:27.979280+0000","last_fullsized":"2024-04-28T12:37:27.979280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:53.650061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.846043+0000","last_change":"2024-04-28T12:37:26.868136+0000","last_active":"2024-04-28T12:37:27.846043+0000","last_peered":"2024-04-28T12:37:27.846043+0000","last_clean":"2024-04-28T12:37:27.846043+0000","last_became_active":"2024-04-28T12:37:26.867663+0000","last_became_peered":"2024-04-28T12:37:26.867663+0000","last_unstale":"2024-04-28T12:37:27.846043+0000","last_undegraded":"2024-04-28T12:37:27.846043+0000","last_fullsized":"2024-04-28T12:37:27.846043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:45.428951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978823+0000","last_change":"2024-04-28T12:37:22.797968+0000","last_active":"2024-04-28T12:37:27.978823+0000","last_peered":"2024-04-28T12:37:27.978823+0000","last_clean":"2024-04-28T12:37:27.978823+0000","last_became_active":"2024-04-28T12:37:22.797258+0000","last_became_peered":"2024-04-28T12:37:22.797258+0000","last_unstale":"2024-04-28T12:37:27.978823+0000","last_undegraded":"2024-04-28T12:37:27.978823+0000","last_fullsized":"2024-04-28T12:37:27.978823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843883+0000","last_change":"2024-04-28T12:37:21.814135+0000","last_active":"2024-04-28T12:37:27.843883+0000","last_peered":"2024-04-28T12:37:27.843883+0000","last_clean":"2024-04-28T12:37:27.843883+0000","last_became_active":"2024-04-28T12:37:21.813800+0000","last_became_peered":"2024-04-28T12:37:21.813800+0000","last_unstale":"2024-04-28T12:37:27.843883+0000","last_undegraded":"2024-04-28T12:37:27.843883+0000","last_fullsized":"2024-04-28T12:37:27.843883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:49.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844709+0000","last_change":"2024-04-28T12:37:26.885109+0000","last_active":"2024-04-28T12:37:27.844709+0000","last_peered":"2024-04-28T12:37:27.844709+0000","last_clean":"2024-04-28T12:37:27.844709+0000","last_became_active":"2024-04-28T12:37:26.882922+0000","last_became_peered":"2024-04-28T12:37:26.882922+0000","last_unstale":"2024-04-28T12:37:27.844709+0000","last_undegraded":"2024-04-28T12:37:27.844709+0000","last_fullsized":"2024-04-28T12:37:27.844709+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:20.147514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978601+0000","last_change":"2024-04-28T12:37:22.802331+0000","last_active":"2024-04-28T12:37:27.978601+0000","last_peered":"2024-04-28T12:37:27.978601+0000","last_clean":"2024-04-28T12:37:27.978601+0000","last_became_active":"2024-04-28T12:37:22.801939+0000","last_became_peered":"2024-04-28T12:37:22.801939+0000","last_unstale":"2024-04-28T12:37:27.978601+0000","last_undegraded":"2024-04-28T12:37:27.978601+0000","last_fullsized":"2024-04-28T12:37:27.978601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:59.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976553+0000","last_change":"2024-04-28T12:37:21.796110+0000","last_active":"2024-04-28T12:37:27.976553+0000","last_peered":"2024-04-28T12:37:27.976553+0000","last_clean":"2024-04-28T12:37:27.976553+0000","last_became_active":"2024-04-28T12:37:21.795405+0000","last_became_peered":"2024-04-28T12:37:21.795405+0000","last_unstale":"2024-04-28T12:37:27.976553+0000","last_undegraded":"2024-04-28T12:37:27.976553+0000","last_fullsized":"2024-04-28T12:37:27.976553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:53.650061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976613+0000","last_change":"2024-04-28T12:37:26.883160+0000","last_active":"2024-04-28T12:37:27.976613+0000","last_peered":"2024-04-28T12:37:27.976613+0000","last_clean":"2024-04-28T12:37:27.976613+0000","last_became_active":"2024-04-28T12:37:26.882684+0000","last_became_peered":"2024-04-28T12:37:26.882684+0000","last_unstale":"2024-04-28T12:37:27.976613+0000","last_undegraded":"2024-04-28T12:37:27.976613+0000","last_fullsized":"2024-04-28T12:37:27.976613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:19:01.590267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981129+0000","last_change":"2024-04-28T12:37:22.800581+0000","last_active":"2024-04-28T12:37:27.981129+0000","last_peered":"2024-04-28T12:37:27.981129+0000","last_clean":"2024-04-28T12:37:27.981129+0000","last_became_active":"2024-04-28T12:37:22.800241+0000","last_became_peered":"2024-04-28T12:37:22.800241+0000","last_unstale":"2024-04-28T12:37:27.981129+0000","last_undegraded":"2024-04-28T12:37:27.981129+0000","last_fullsized":"2024-04-28T12:37:27.981129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982900+0000","last_change":"2024-04-28T12:37:21.811668+0000","last_active":"2024-04-28T12:37:27.982900+0000","last_peered":"2024-04-28T12:37:27.982900+0000","last_clean":"2024-04-28T12:37:27.982900+0000","last_became_active":"2024-04-28T12:37:21.811372+0000","last_became_peered":"2024-04-28T12:37:21.811372+0000","last_unstale":"2024-04-28T12:37:27.982900+0000","last_undegraded":"2024-04-28T12:37:27.982900+0000","last_fullsized":"2024-04-28T12:37:27.982900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:00:04.591555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976863+0000","last_change":"2024-04-28T12:37:26.877753+0000","last_active":"2024-04-28T12:37:27.976863+0000","last_peered":"2024-04-28T12:37:27.976863+0000","last_clean":"2024-04-28T12:37:27.976863+0000","last_became_active":"2024-04-28T12:37:26.877456+0000","last_became_peered":"2024-04-28T12:37:26.877456+0000","last_unstale":"2024-04-28T12:37:27.976863+0000","last_undegraded":"2024-04-28T12:37:27.976863+0000","last_fullsized":"2024-04-28T12:37:27.976863+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:45.428951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979882+0000","last_change":"2024-04-28T12:37:22.801683+0000","last_active":"2024-04-28T12:37:27.979882+0000","last_peered":"2024-04-28T12:37:27.979882+0000","last_clean":"2024-04-28T12:37:27.979882+0000","last_became_active":"2024-04-28T12:37:22.801378+0000","last_became_peered":"2024-04-28T12:37:22.801378+0000","last_unstale":"2024-04-28T12:37:27.979882+0000","last_undegraded":"2024-04-28T12:37:27.979882+0000","last_fullsized":"2024-04-28T12:37:27.979882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:41.371396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981252+0000","last_change":"2024-04-28T12:37:21.791074+0000","last_active":"2024-04-28T12:37:27.981252+0000","last_peered":"2024-04-28T12:37:27.981252+0000","last_clean":"2024-04-28T12:37:27.981252+0000","last_became_active":"2024-04-28T12:37:21.790645+0000","last_became_peered":"2024-04-28T12:37:21.790645+0000","last_unstale":"2024-04-28T12:37:27.981252+0000","last_undegraded":"2024-04-28T12:37:27.981252+0000","last_fullsized":"2024-04-28T12:37:27.981252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977495+0000","last_change":"2024-04-28T12:37:26.873652+0000","last_active":"2024-04-28T12:37:27.977495+0000","last_peered":"2024-04-28T12:37:27.977495+0000","last_clean":"2024-04-28T12:37:27.977495+0000","last_became_active":"2024-04-28T12:37:26.873065+0000","last_became_peered":"2024-04-28T12:37:26.873065+0000","last_unstale":"2024-04-28T12:37:27.977495+0000","last_undegraded":"2024-04-28T12:37:27.977495+0000","last_fullsized":"2024-04-28T12:37:27.977495+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845437+0000","last_change":"2024-04-28T12:37:22.794106+0000","last_active":"2024-04-28T12:37:27.845437+0000","last_peered":"2024-04-28T12:37:27.845437+0000","last_clean":"2024-04-28T12:37:27.845437+0000","last_became_active":"2024-04-28T12:37:22.789651+0000","last_became_peered":"2024-04-28T12:37:22.789651+0000","last_unstale":"2024-04-28T12:37:27.845437+0000","last_undegraded":"2024-04-28T12:37:27.845437+0000","last_fullsized":"2024-04-28T12:37:27.845437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:33.269962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979186+0000","last_change":"2024-04-28T12:37:21.810116+0000","last_active":"2024-04-28T12:37:27.979186+0000","last_peered":"2024-04-28T12:37:27.979186+0000","last_clean":"2024-04-28T12:37:27.979186+0000","last_became_active":"2024-04-28T12:37:21.809736+0000","last_became_peered":"2024-04-28T12:37:21.809736+0000","last_unstale":"2024-04-28T12:37:27.979186+0000","last_undegraded":"2024-04-28T12:37:27.979186+0000","last_fullsized":"2024-04-28T12:37:27.979186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:39.334819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981752+0000","last_change":"2024-04-28T12:37:26.867239+0000","last_active":"2024-04-28T12:37:27.981752+0000","last_peered":"2024-04-28T12:37:27.981752+0000","last_clean":"2024-04-28T12:37:27.981752+0000","last_became_active":"2024-04-28T12:37:26.866848+0000","last_became_peered":"2024-04-28T12:37:26.866848+0000","last_unstale":"2024-04-28T12:37:27.981752+0000","last_undegraded":"2024-04-28T12:37:27.981752+0000","last_fullsized":"2024-04-28T12:37:27.981752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:08:51.032543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976301+0000","last_change":"2024-04-28T12:37:22.796512+0000","last_active":"2024-04-28T12:37:27.976301+0000","last_peered":"2024-04-28T12:37:27.976301+0000","last_clean":"2024-04-28T12:37:27.976301+0000","last_became_active":"2024-04-28T12:37:22.793368+0000","last_became_peered":"2024-04-28T12:37:22.793368+0000","last_unstale":"2024-04-28T12:37:27.976301+0000","last_undegraded":"2024-04-28T12:37:27.976301+0000","last_fullsized":"2024-04-28T12:37:27.976301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.839137+0000","last_change":"2024-04-28T12:37:21.793644+0000","last_active":"2024-04-28T12:37:27.839137+0000","last_peered":"2024-04-28T12:37:27.839137+0000","last_clean":"2024-04-28T12:37:27.839137+0000","last_became_active":"2024-04-28T12:37:21.793358+0000","last_became_peered":"2024-04-28T12:37:21.793358+0000","last_unstale":"2024-04-28T12:37:27.839137+0000","last_undegraded":"2024-04-28T12:37:27.839137+0000","last_fullsized":"2024-04-28T12:37:27.839137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:55:28.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978474+0000","last_change":"2024-04-28T12:37:26.866571+0000","last_active":"2024-04-28T12:37:27.978474+0000","last_peered":"2024-04-28T12:37:27.978474+0000","last_clean":"2024-04-28T12:37:27.978474+0000","last_became_active":"2024-04-28T12:37:26.866042+0000","last_became_peered":"2024-04-28T12:37:26.866042+0000","last_unstale":"2024-04-28T12:37:27.978474+0000","last_undegraded":"2024-04-28T12:37:27.978474+0000","last_fullsized":"2024-04-28T12:37:27.978474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:04.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976223+0000","last_change":"2024-04-28T12:37:22.796747+0000","last_active":"2024-04-28T12:37:27.976223+0000","last_peered":"2024-04-28T12:37:27.976223+0000","last_clean":"2024-04-28T12:37:27.976223+0000","last_became_active":"2024-04-28T12:37:22.795677+0000","last_became_peered":"2024-04-28T12:37:22.795677+0000","last_unstale":"2024-04-28T12:37:27.976223+0000","last_undegraded":"2024-04-28T12:37:27.976223+0000","last_fullsized":"2024-04-28T12:37:27.976223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:05:18.644675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979595+0000","last_change":"2024-04-28T12:37:21.814725+0000","last_active":"2024-04-28T12:37:27.979595+0000","last_peered":"2024-04-28T12:37:27.979595+0000","last_clean":"2024-04-28T12:37:27.979595+0000","last_became_active":"2024-04-28T12:37:21.814516+0000","last_became_peered":"2024-04-28T12:37:21.814516+0000","last_unstale":"2024-04-28T12:37:27.979595+0000","last_undegraded":"2024-04-28T12:37:27.979595+0000","last_fullsized":"2024-04-28T12:37:27.979595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:49.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978718+0000","last_change":"2024-04-28T12:37:26.886450+0000","last_active":"2024-04-28T12:37:27.978718+0000","last_peered":"2024-04-28T12:37:27.978718+0000","last_clean":"2024-04-28T12:37:27.978718+0000","last_became_active":"2024-04-28T12:37:26.883398+0000","last_became_peered":"2024-04-28T12:37:26.883398+0000","last_unstale":"2024-04-28T12:37:27.978718+0000","last_undegraded":"2024-04-28T12:37:27.978718+0000","last_fullsized":"2024-04-28T12:37:27.978718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:04.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977534+0000","last_change":"2024-04-28T12:37:22.814159+0000","last_active":"2024-04-28T12:37:27.977534+0000","last_peered":"2024-04-28T12:37:27.977534+0000","last_clean":"2024-04-28T12:37:27.977534+0000","last_became_active":"2024-04-28T12:37:22.813889+0000","last_became_peered":"2024-04-28T12:37:22.813889+0000","last_unstale":"2024-04-28T12:37:27.977534+0000","last_undegraded":"2024-04-28T12:37:27.977534+0000","last_fullsized":"2024-04-28T12:37:27.977534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:16.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979063+0000","last_change":"2024-04-28T12:37:21.803169+0000","last_active":"2024-04-28T12:37:27.979063+0000","last_peered":"2024-04-28T12:37:27.979063+0000","last_clean":"2024-04-28T12:37:27.979063+0000","last_became_active":"2024-04-28T12:37:21.800812+0000","last_became_peered":"2024-04-28T12:37:21.800812+0000","last_unstale":"2024-04-28T12:37:27.979063+0000","last_undegraded":"2024-04-28T12:37:27.979063+0000","last_fullsized":"2024-04-28T12:37:27.979063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:55:28.785820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979702+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:37:27.979702+0000","last_peered":"2024-04-28T12:37:27.979702+0000","last_clean":"2024-04-28T12:37:27.979702+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:37:27.979702+0000","last_undegraded":"2024-04-28T12:37:27.979702+0000","last_fullsized":"2024-04-28T12:37:27.979702+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843882+0000","last_change":"2024-04-28T12:37:26.884731+0000","last_active":"2024-04-28T12:37:27.843882+0000","last_peered":"2024-04-28T12:37:27.843882+0000","last_clean":"2024-04-28T12:37:27.843882+0000","last_became_active":"2024-04-28T12:37:26.882799+0000","last_became_peered":"2024-04-28T12:37:26.882799+0000","last_unstale":"2024-04-28T12:37:27.843882+0000","last_undegraded":"2024-04-28T12:37:27.843882+0000","last_fullsized":"2024-04-28T12:37:27.843882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:51.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982421+0000","last_change":"2024-04-28T12:37:22.818975+0000","last_active":"2024-04-28T12:37:27.982421+0000","last_peered":"2024-04-28T12:37:27.982421+0000","last_clean":"2024-04-28T12:37:27.982421+0000","last_became_active":"2024-04-28T12:37:22.818677+0000","last_became_peered":"2024-04-28T12:37:22.818677+0000","last_unstale":"2024-04-28T12:37:27.982421+0000","last_undegraded":"2024-04-28T12:37:27.982421+0000","last_fullsized":"2024-04-28T12:37:27.982421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:41.371396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840221+0000","last_change":"2024-04-28T12:37:21.800771+0000","last_active":"2024-04-28T12:37:27.840221+0000","last_peered":"2024-04-28T12:37:27.840221+0000","last_clean":"2024-04-28T12:37:27.840221+0000","last_became_active":"2024-04-28T12:37:21.800451+0000","last_became_peered":"2024-04-28T12:37:21.800451+0000","last_unstale":"2024-04-28T12:37:27.840221+0000","last_undegraded":"2024-04-28T12:37:27.840221+0000","last_fullsized":"2024-04-28T12:37:27.840221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842666+0000","last_change":"2024-04-28T12:37:26.879284+0000","last_active":"2024-04-28T12:37:27.842666+0000","last_peered":"2024-04-28T12:37:27.842666+0000","last_clean":"2024-04-28T12:37:27.842666+0000","last_became_active":"2024-04-28T12:37:26.878864+0000","last_became_peered":"2024-04-28T12:37:26.878864+0000","last_unstale":"2024-04-28T12:37:27.842666+0000","last_undegraded":"2024-04-28T12:37:27.842666+0000","last_fullsized":"2024-04-28T12:37:27.842666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:47:45.428951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979686+0000","last_change":"2024-04-28T12:37:22.824563+0000","last_active":"2024-04-28T12:37:27.979686+0000","last_peered":"2024-04-28T12:37:27.979686+0000","last_clean":"2024-04-28T12:37:27.979686+0000","last_became_active":"2024-04-28T12:37:22.818493+0000","last_became_peered":"2024-04-28T12:37:22.818493+0000","last_unstale":"2024-04-28T12:37:27.979686+0000","last_undegraded":"2024-04-28T12:37:27.979686+0000","last_fullsized":"2024-04-28T12:37:27.979686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:00.502362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976419+0000","last_change":"2024-04-28T12:37:21.799906+0000","last_active":"2024-04-28T12:37:27.976419+0000","last_peered":"2024-04-28T12:37:27.976419+0000","last_clean":"2024-04-28T12:37:27.976419+0000","last_became_active":"2024-04-28T12:37:21.799470+0000","last_became_peered":"2024-04-28T12:37:21.799470+0000","last_unstale":"2024-04-28T12:37:27.976419+0000","last_undegraded":"2024-04-28T12:37:27.976419+0000","last_fullsized":"2024-04-28T12:37:27.976419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:49.687780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977214+0000","last_change":"2024-04-28T12:37:26.874939+0000","last_active":"2024-04-28T12:37:27.977214+0000","last_peered":"2024-04-28T12:37:27.977214+0000","last_clean":"2024-04-28T12:37:27.977214+0000","last_became_active":"2024-04-28T12:37:26.874472+0000","last_became_peered":"2024-04-28T12:37:26.874472+0000","last_unstale":"2024-04-28T12:37:27.977214+0000","last_undegraded":"2024-04-28T12:37:27.977214+0000","last_fullsized":"2024-04-28T12:37:27.977214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981950+0000","last_change":"2024-04-28T12:37:22.814584+0000","last_active":"2024-04-28T12:37:27.981950+0000","last_peered":"2024-04-28T12:37:27.981950+0000","last_clean":"2024-04-28T12:37:27.981950+0000","last_became_active":"2024-04-28T12:37:22.812752+0000","last_became_peered":"2024-04-28T12:37:22.812752+0000","last_unstale":"2024-04-28T12:37:27.981950+0000","last_undegraded":"2024-04-28T12:37:27.981950+0000","last_fullsized":"2024-04-28T12:37:27.981950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:48:32.880830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840416+0000","last_change":"2024-04-28T12:37:21.801322+0000","last_active":"2024-04-28T12:37:27.840416+0000","last_peered":"2024-04-28T12:37:27.840416+0000","last_clean":"2024-04-28T12:37:27.840416+0000","last_became_active":"2024-04-28T12:37:21.800437+0000","last_became_peered":"2024-04-28T12:37:21.800437+0000","last_unstale":"2024-04-28T12:37:27.840416+0000","last_undegraded":"2024-04-28T12:37:27.840416+0000","last_fullsized":"2024-04-28T12:37:27.840416+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:33.616030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978163+0000","last_change":"2024-04-28T12:37:26.885854+0000","last_active":"2024-04-28T12:37:27.978163+0000","last_peered":"2024-04-28T12:37:27.978163+0000","last_clean":"2024-04-28T12:37:27.978163+0000","last_became_active":"2024-04-28T12:37:26.884298+0000","last_became_peered":"2024-04-28T12:37:26.884298+0000","last_unstale":"2024-04-28T12:37:27.978163+0000","last_undegraded":"2024-04-28T12:37:27.978163+0000","last_fullsized":"2024-04-28T12:37:27.978163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:19:01.590267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982107+0000","last_change":"2024-04-28T12:37:22.808698+0000","last_active":"2024-04-28T12:37:27.982107+0000","last_peered":"2024-04-28T12:37:27.982107+0000","last_clean":"2024-04-28T12:37:27.982107+0000","last_became_active":"2024-04-28T12:37:22.808363+0000","last_became_peered":"2024-04-28T12:37:22.808363+0000","last_unstale":"2024-04-28T12:37:27.982107+0000","last_undegraded":"2024-04-28T12:37:27.982107+0000","last_fullsized":"2024-04-28T12:37:27.982107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:47.618755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.846296+0000","last_change":"2024-04-28T12:37:21.794490+0000","last_active":"2024-04-28T12:37:27.846296+0000","last_peered":"2024-04-28T12:37:27.846296+0000","last_clean":"2024-04-28T12:37:27.846296+0000","last_became_active":"2024-04-28T12:37:21.794115+0000","last_became_peered":"2024-04-28T12:37:21.794115+0000","last_unstale":"2024-04-28T12:37:27.846296+0000","last_undegraded":"2024-04-28T12:37:27.846296+0000","last_fullsized":"2024-04-28T12:37:27.846296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:20:04.188478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844144+0000","last_change":"2024-04-28T12:37:26.867905+0000","last_active":"2024-04-28T12:37:27.844144+0000","last_peered":"2024-04-28T12:37:27.844144+0000","last_clean":"2024-04-28T12:37:27.844144+0000","last_became_active":"2024-04-28T12:37:26.867317+0000","last_became_peered":"2024-04-28T12:37:26.867317+0000","last_unstale":"2024-04-28T12:37:27.844144+0000","last_undegraded":"2024-04-28T12:37:27.844144+0000","last_fullsized":"2024-04-28T12:37:27.844144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:49.659072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980575+0000","last_change":"2024-04-28T12:37:22.802406+0000","last_active":"2024-04-28T12:37:27.980575+0000","last_peered":"2024-04-28T12:37:27.980575+0000","last_clean":"2024-04-28T12:37:27.980575+0000","last_became_active":"2024-04-28T12:37:22.802094+0000","last_became_peered":"2024-04-28T12:37:22.802094+0000","last_unstale":"2024-04-28T12:37:27.980575+0000","last_undegraded":"2024-04-28T12:37:27.980575+0000","last_fullsized":"2024-04-28T12:37:27.980575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:16.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":29,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.838661+0000","last_change":"2024-04-28T12:37:21.788140+0000","last_active":"2024-04-28T12:37:27.838661+0000","last_peered":"2024-04-28T12:37:27.838661+0000","last_clean":"2024-04-28T12:37:27.838661+0000","last_became_active":"2024-04-28T12:37:21.787727+0000","last_became_peered":"2024-04-28T12:37:21.787727+0000","last_unstale":"2024-04-28T12:37:27.838661+0000","last_undegraded":"2024-04-28T12:37:27.838661+0000","last_fullsized":"2024-04-28T12:37:27.838661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:29:20.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978548+0000","last_change":"2024-04-28T12:37:26.868358+0000","last_active":"2024-04-28T12:37:27.978548+0000","last_peered":"2024-04-28T12:37:27.978548+0000","last_clean":"2024-04-28T12:37:27.978548+0000","last_became_active":"2024-04-28T12:37:26.867972+0000","last_became_peered":"2024-04-28T12:37:26.867972+0000","last_unstale":"2024-04-28T12:37:27.978548+0000","last_undegraded":"2024-04-28T12:37:27.978548+0000","last_fullsized":"2024-04-28T12:37:27.978548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:28.885007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842120+0000","last_change":"2024-04-28T12:37:22.786192+0000","last_active":"2024-04-28T12:37:27.842120+0000","last_peered":"2024-04-28T12:37:27.842120+0000","last_clean":"2024-04-28T12:37:27.842120+0000","last_became_active":"2024-04-28T12:37:22.785730+0000","last_became_peered":"2024-04-28T12:37:22.785730+0000","last_unstale":"2024-04-28T12:37:27.842120+0000","last_undegraded":"2024-04-28T12:37:27.842120+0000","last_fullsized":"2024-04-28T12:37:27.842120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:42.893320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842909+0000","last_change":"2024-04-28T12:37:21.800159+0000","last_active":"2024-04-28T12:37:27.842909+0000","last_peered":"2024-04-28T12:37:27.842909+0000","last_clean":"2024-04-28T12:37:27.842909+0000","last_became_active":"2024-04-28T12:37:21.799817+0000","last_became_peered":"2024-04-28T12:37:21.799817+0000","last_unstale":"2024-04-28T12:37:27.842909+0000","last_undegraded":"2024-04-28T12:37:27.842909+0000","last_fullsized":"2024-04-28T12:37:27.842909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978507+0000","last_change":"2024-04-28T12:37:26.883264+0000","last_active":"2024-04-28T12:37:27.978507+0000","last_peered":"2024-04-28T12:37:27.978507+0000","last_clean":"2024-04-28T12:37:27.978507+0000","last_became_active":"2024-04-28T12:37:26.882787+0000","last_became_peered":"2024-04-28T12:37:26.882787+0000","last_unstale":"2024-04-28T12:37:27.978507+0000","last_undegraded":"2024-04-28T12:37:27.978507+0000","last_fullsized":"2024-04-28T12:37:27.978507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:11:24.219747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981395+0000","last_change":"2024-04-28T12:37:22.811865+0000","last_active":"2024-04-28T12:37:27.981395+0000","last_peered":"2024-04-28T12:37:27.981395+0000","last_clean":"2024-04-28T12:37:27.981395+0000","last_became_active":"2024-04-28T12:37:22.809479+0000","last_became_peered":"2024-04-28T12:37:22.809479+0000","last_unstale":"2024-04-28T12:37:27.981395+0000","last_undegraded":"2024-04-28T12:37:27.981395+0000","last_fullsized":"2024-04-28T12:37:27.981395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:39.958329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978873+0000","last_change":"2024-04-28T12:37:21.805620+0000","last_active":"2024-04-28T12:37:27.978873+0000","last_peered":"2024-04-28T12:37:27.978873+0000","last_clean":"2024-04-28T12:37:27.978873+0000","last_became_active":"2024-04-28T12:37:21.805397+0000","last_became_peered":"2024-04-28T12:37:21.805397+0000","last_unstale":"2024-04-28T12:37:27.978873+0000","last_undegraded":"2024-04-28T12:37:27.978873+0000","last_fullsized":"2024-04-28T12:37:27.978873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:49:17.762752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978474+0000","last_change":"2024-04-28T12:37:26.880544+0000","last_active":"2024-04-28T12:37:27.978474+0000","last_peered":"2024-04-28T12:37:27.978474+0000","last_clean":"2024-04-28T12:37:27.978474+0000","last_became_active":"2024-04-28T12:37:26.878960+0000","last_became_peered":"2024-04-28T12:37:26.878960+0000","last_unstale":"2024-04-28T12:37:27.978474+0000","last_undegraded":"2024-04-28T12:37:27.978474+0000","last_fullsized":"2024-04-28T12:37:27.978474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:28:21.376871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977145+0000","last_change":"2024-04-28T12:37:22.795607+0000","last_active":"2024-04-28T12:37:27.977145+0000","last_peered":"2024-04-28T12:37:27.977145+0000","last_clean":"2024-04-28T12:37:27.977145+0000","last_became_active":"2024-04-28T12:37:22.795262+0000","last_became_peered":"2024-04-28T12:37:22.795262+0000","last_unstale":"2024-04-28T12:37:27.977145+0000","last_undegraded":"2024-04-28T12:37:27.977145+0000","last_fullsized":"2024-04-28T12:37:27.977145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:42.893320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977438+0000","last_change":"2024-04-28T12:37:21.798231+0000","last_active":"2024-04-28T12:37:27.977438+0000","last_peered":"2024-04-28T12:37:27.977438+0000","last_clean":"2024-04-28T12:37:27.977438+0000","last_became_active":"2024-04-28T12:37:21.797978+0000","last_became_peered":"2024-04-28T12:37:21.797978+0000","last_unstale":"2024-04-28T12:37:27.977438+0000","last_undegraded":"2024-04-28T12:37:27.977438+0000","last_fullsized":"2024-04-28T12:37:27.977438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:20:04.188478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980535+0000","last_change":"2024-04-28T12:37:26.881273+0000","last_active":"2024-04-28T12:37:27.980535+0000","last_peered":"2024-04-28T12:37:27.980535+0000","last_clean":"2024-04-28T12:37:27.980535+0000","last_became_active":"2024-04-28T12:37:26.880845+0000","last_became_peered":"2024-04-28T12:37:26.880845+0000","last_unstale":"2024-04-28T12:37:27.980535+0000","last_undegraded":"2024-04-28T12:37:27.980535+0000","last_fullsized":"2024-04-28T12:37:27.980535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:44:09.634939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981232+0000","last_change":"2024-04-28T12:37:22.811901+0000","last_active":"2024-04-28T12:37:27.981232+0000","last_peered":"2024-04-28T12:37:27.981232+0000","last_clean":"2024-04-28T12:37:27.981232+0000","last_became_active":"2024-04-28T12:37:22.807998+0000","last_became_peered":"2024-04-28T12:37:22.807998+0000","last_unstale":"2024-04-28T12:37:27.981232+0000","last_undegraded":"2024-04-28T12:37:27.981232+0000","last_fullsized":"2024-04-28T12:37:27.981232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:29:25.559764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:41.723137+0000","last_change":"2024-04-28T12:37:21.802895+0000","last_active":"2024-04-28T12:37:41.723137+0000","last_peered":"2024-04-28T12:37:41.723137+0000","last_clean":"2024-04-28T12:37:41.723137+0000","last_became_active":"2024-04-28T12:37:21.801912+0000","last_became_peered":"2024-04-28T12:37:21.801912+0000","last_unstale":"2024-04-28T12:37:41.723137+0000","last_undegraded":"2024-04-28T12:37:41.723137+0000","last_fullsized":"2024-04-28T12:37:41.723137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:39.334819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977722+0000","last_change":"2024-04-28T12:37:26.873444+0000","last_active":"2024-04-28T12:37:27.977722+0000","last_peered":"2024-04-28T12:37:27.977722+0000","last_clean":"2024-04-28T12:37:27.977722+0000","last_became_active":"2024-04-28T12:37:26.872331+0000","last_became_peered":"2024-04-28T12:37:26.872331+0000","last_unstale":"2024-04-28T12:37:27.977722+0000","last_undegraded":"2024-04-28T12:37:27.977722+0000","last_fullsized":"2024-04-28T12:37:27.977722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:51.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845884+0000","last_change":"2024-04-28T12:37:22.786473+0000","last_active":"2024-04-28T12:37:27.845884+0000","last_peered":"2024-04-28T12:37:27.845884+0000","last_clean":"2024-04-28T12:37:27.845884+0000","last_became_active":"2024-04-28T12:37:22.785266+0000","last_became_peered":"2024-04-28T12:37:22.785266+0000","last_unstale":"2024-04-28T12:37:27.845884+0000","last_undegraded":"2024-04-28T12:37:27.845884+0000","last_fullsized":"2024-04-28T12:37:27.845884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:42:50.704472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976275+0000","last_change":"2024-04-28T12:37:21.810514+0000","last_active":"2024-04-28T12:37:27.976275+0000","last_peered":"2024-04-28T12:37:27.976275+0000","last_clean":"2024-04-28T12:37:27.976275+0000","last_became_active":"2024-04-28T12:37:21.810221+0000","last_became_peered":"2024-04-28T12:37:21.810221+0000","last_unstale":"2024-04-28T12:37:27.976275+0000","last_undegraded":"2024-04-28T12:37:27.976275+0000","last_fullsized":"2024-04-28T12:37:27.976275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845261+0000","last_change":"2024-04-28T12:37:26.871384+0000","last_active":"2024-04-28T12:37:27.845261+0000","last_peered":"2024-04-28T12:37:27.845261+0000","last_clean":"2024-04-28T12:37:27.845261+0000","last_became_active":"2024-04-28T12:37:26.868701+0000","last_became_peered":"2024-04-28T12:37:26.868701+0000","last_unstale":"2024-04-28T12:37:27.845261+0000","last_undegraded":"2024-04-28T12:37:27.845261+0000","last_fullsized":"2024-04-28T12:37:27.845261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:26:44.015884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979855+0000","last_change":"2024-04-28T12:37:22.821706+0000","last_active":"2024-04-28T12:37:27.979855+0000","last_peered":"2024-04-28T12:37:27.979855+0000","last_clean":"2024-04-28T12:37:27.979855+0000","last_became_active":"2024-04-28T12:37:22.821338+0000","last_became_peered":"2024-04-28T12:37:22.821338+0000","last_unstale":"2024-04-28T12:37:27.979855+0000","last_undegraded":"2024-04-28T12:37:27.979855+0000","last_fullsized":"2024-04-28T12:37:27.979855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980054+0000","last_change":"2024-04-28T12:37:21.803535+0000","last_active":"2024-04-28T12:37:27.980054+0000","last_peered":"2024-04-28T12:37:27.980054+0000","last_clean":"2024-04-28T12:37:27.980054+0000","last_became_active":"2024-04-28T12:37:21.801895+0000","last_became_peered":"2024-04-28T12:37:21.801895+0000","last_unstale":"2024-04-28T12:37:27.980054+0000","last_undegraded":"2024-04-28T12:37:27.980054+0000","last_fullsized":"2024-04-28T12:37:27.980054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:32.253270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976957+0000","last_change":"2024-04-28T12:37:26.861154+0000","last_active":"2024-04-28T12:37:27.976957+0000","last_peered":"2024-04-28T12:37:27.976957+0000","last_clean":"2024-04-28T12:37:27.976957+0000","last_became_active":"2024-04-28T12:37:26.860732+0000","last_became_peered":"2024-04-28T12:37:26.860732+0000","last_unstale":"2024-04-28T12:37:27.976957+0000","last_undegraded":"2024-04-28T12:37:27.976957+0000","last_fullsized":"2024-04-28T12:37:27.976957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:48:40.360635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979341+0000","last_change":"2024-04-28T12:37:22.817129+0000","last_active":"2024-04-28T12:37:27.979341+0000","last_peered":"2024-04-28T12:37:27.979341+0000","last_clean":"2024-04-28T12:37:27.979341+0000","last_became_active":"2024-04-28T12:37:22.816811+0000","last_became_peered":"2024-04-28T12:37:22.816811+0000","last_unstale":"2024-04-28T12:37:27.979341+0000","last_undegraded":"2024-04-28T12:37:27.979341+0000","last_fullsized":"2024-04-28T12:37:27.979341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:29:25.559764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843391+0000","last_change":"2024-04-28T12:37:21.805873+0000","last_active":"2024-04-28T12:37:27.843391+0000","last_peered":"2024-04-28T12:37:27.843391+0000","last_clean":"2024-04-28T12:37:27.843391+0000","last_became_active":"2024-04-28T12:37:21.805549+0000","last_became_peered":"2024-04-28T12:37:21.805549+0000","last_unstale":"2024-04-28T12:37:27.843391+0000","last_undegraded":"2024-04-28T12:37:27.843391+0000","last_fullsized":"2024-04-28T12:37:27.843391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976927+0000","last_change":"2024-04-28T12:37:26.883208+0000","last_active":"2024-04-28T12:37:27.976927+0000","last_peered":"2024-04-28T12:37:27.976927+0000","last_clean":"2024-04-28T12:37:27.976927+0000","last_became_active":"2024-04-28T12:37:26.882732+0000","last_became_peered":"2024-04-28T12:37:26.882732+0000","last_unstale":"2024-04-28T12:37:27.976927+0000","last_undegraded":"2024-04-28T12:37:27.976927+0000","last_fullsized":"2024-04-28T12:37:27.976927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:37:09.856024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978306+0000","last_change":"2024-04-28T12:37:26.888160+0000","last_active":"2024-04-28T12:37:27.978306+0000","last_peered":"2024-04-28T12:37:27.978306+0000","last_clean":"2024-04-28T12:37:27.978306+0000","last_became_active":"2024-04-28T12:37:26.886631+0000","last_became_peered":"2024-04-28T12:37:26.886631+0000","last_unstale":"2024-04-28T12:37:27.978306+0000","last_undegraded":"2024-04-28T12:37:27.978306+0000","last_fullsized":"2024-04-28T12:37:27.978306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:51.751437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.975748+0000","last_change":"2024-04-28T12:37:22.794465+0000","last_active":"2024-04-28T12:37:27.975748+0000","last_peered":"2024-04-28T12:37:27.975748+0000","last_clean":"2024-04-28T12:37:27.975748+0000","last_became_active":"2024-04-28T12:37:22.794138+0000","last_became_peered":"2024-04-28T12:37:22.794138+0000","last_unstale":"2024-04-28T12:37:27.975748+0000","last_undegraded":"2024-04-28T12:37:27.975748+0000","last_fullsized":"2024-04-28T12:37:27.975748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978354+0000","last_change":"2024-04-28T12:37:21.805697+0000","last_active":"2024-04-28T12:37:27.978354+0000","last_peered":"2024-04-28T12:37:27.978354+0000","last_clean":"2024-04-28T12:37:27.978354+0000","last_became_active":"2024-04-28T12:37:21.805449+0000","last_became_peered":"2024-04-28T12:37:21.805449+0000","last_unstale":"2024-04-28T12:37:27.978354+0000","last_undegraded":"2024-04-28T12:37:27.978354+0000","last_fullsized":"2024-04-28T12:37:27.978354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:06:59.502811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843785+0000","last_change":"2024-04-28T12:37:26.869931+0000","last_active":"2024-04-28T12:37:27.843785+0000","last_peered":"2024-04-28T12:37:27.843785+0000","last_clean":"2024-04-28T12:37:27.843785+0000","last_became_active":"2024-04-28T12:37:26.869441+0000","last_became_peered":"2024-04-28T12:37:26.869441+0000","last_unstale":"2024-04-28T12:37:27.843785+0000","last_undegraded":"2024-04-28T12:37:27.843785+0000","last_fullsized":"2024-04-28T12:37:27.843785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:51.676310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977035+0000","last_change":"2024-04-28T12:37:22.792490+0000","last_active":"2024-04-28T12:37:27.977035+0000","last_peered":"2024-04-28T12:37:27.977035+0000","last_clean":"2024-04-28T12:37:27.977035+0000","last_became_active":"2024-04-28T12:37:22.790799+0000","last_became_peered":"2024-04-28T12:37:22.790799+0000","last_unstale":"2024-04-28T12:37:27.977035+0000","last_undegraded":"2024-04-28T12:37:27.977035+0000","last_fullsized":"2024-04-28T12:37:27.977035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977471+0000","last_change":"2024-04-28T12:37:21.799595+0000","last_active":"2024-04-28T12:37:27.977471+0000","last_peered":"2024-04-28T12:37:27.977471+0000","last_clean":"2024-04-28T12:37:27.977471+0000","last_became_active":"2024-04-28T12:37:21.796500+0000","last_became_peered":"2024-04-28T12:37:21.796500+0000","last_unstale":"2024-04-28T12:37:27.977471+0000","last_undegraded":"2024-04-28T12:37:27.977471+0000","last_fullsized":"2024-04-28T12:37:27.977471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:33.616030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.843252+0000","last_change":"2024-04-28T12:37:21.799592+0000","last_active":"2024-04-28T12:37:27.843252+0000","last_peered":"2024-04-28T12:37:27.843252+0000","last_clean":"2024-04-28T12:37:27.843252+0000","last_became_active":"2024-04-28T12:37:21.799085+0000","last_became_peered":"2024-04-28T12:37:21.799085+0000","last_unstale":"2024-04-28T12:37:27.843252+0000","last_undegraded":"2024-04-28T12:37:27.843252+0000","last_fullsized":"2024-04-28T12:37:27.843252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:53.650061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977307+0000","last_change":"2024-04-28T12:37:22.781856+0000","last_active":"2024-04-28T12:37:27.977307+0000","last_peered":"2024-04-28T12:37:27.977307+0000","last_clean":"2024-04-28T12:37:27.977307+0000","last_became_active":"2024-04-28T12:37:22.780347+0000","last_became_peered":"2024-04-28T12:37:22.780347+0000","last_unstale":"2024-04-28T12:37:27.977307+0000","last_undegraded":"2024-04-28T12:37:27.977307+0000","last_fullsized":"2024-04-28T12:37:27.977307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840176+0000","last_change":"2024-04-28T12:37:26.867761+0000","last_active":"2024-04-28T12:37:27.840176+0000","last_peered":"2024-04-28T12:37:27.840176+0000","last_clean":"2024-04-28T12:37:27.840176+0000","last_became_active":"2024-04-28T12:37:26.867277+0000","last_became_peered":"2024-04-28T12:37:26.867277+0000","last_unstale":"2024-04-28T12:37:27.840176+0000","last_undegraded":"2024-04-28T12:37:27.840176+0000","last_fullsized":"2024-04-28T12:37:27.840176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:12:40.786592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977843+0000","last_change":"2024-04-28T12:37:21.812666+0000","last_active":"2024-04-28T12:37:27.977843+0000","last_peered":"2024-04-28T12:37:27.977843+0000","last_clean":"2024-04-28T12:37:27.977843+0000","last_became_active":"2024-04-28T12:37:21.812384+0000","last_became_peered":"2024-04-28T12:37:21.812384+0000","last_unstale":"2024-04-28T12:37:27.977843+0000","last_undegraded":"2024-04-28T12:37:27.977843+0000","last_fullsized":"2024-04-28T12:37:27.977843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:20.732423+0000","last_clean_scrub_stamp":"2024-04-28T12:37:20.732423+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:29:20.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844555+0000","last_change":"2024-04-28T12:37:22.794127+0000","last_active":"2024-04-28T12:37:27.844555+0000","last_peered":"2024-04-28T12:37:27.844555+0000","last_clean":"2024-04-28T12:37:27.844555+0000","last_became_active":"2024-04-28T12:37:22.793770+0000","last_became_peered":"2024-04-28T12:37:22.793770+0000","last_unstale":"2024-04-28T12:37:27.844555+0000","last_undegraded":"2024-04-28T12:37:27.844555+0000","last_fullsized":"2024-04-28T12:37:27.844555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:00:05.608247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977959+0000","last_change":"2024-04-28T12:37:26.863918+0000","last_active":"2024-04-28T12:37:27.977959+0000","last_peered":"2024-04-28T12:37:27.977959+0000","last_clean":"2024-04-28T12:37:27.977959+0000","last_became_active":"2024-04-28T12:37:26.863558+0000","last_became_peered":"2024-04-28T12:37:26.863558+0000","last_unstale":"2024-04-28T12:37:27.977959+0000","last_undegraded":"2024-04-28T12:37:27.977959+0000","last_fullsized":"2024-04-28T12:37:27.977959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:25.806670+0000","last_clean_scrub_stamp":"2024-04-28T12:37:25.806670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:49:38.590277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.841795+0000","last_change":"2024-04-28T12:37:22.786731+0000","last_active":"2024-04-28T12:37:27.841795+0000","last_peered":"2024-04-28T12:37:27.841795+0000","last_clean":"2024-04-28T12:37:27.841795+0000","last_became_active":"2024-04-28T12:37:22.786283+0000","last_became_peered":"2024-04-28T12:37:22.786283+0000","last_unstale":"2024-04-28T12:37:27.841795+0000","last_undegraded":"2024-04-28T12:37:27.841795+0000","last_fullsized":"2024-04-28T12:37:27.841795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.838968+0000","last_change":"2024-04-28T12:37:22.793953+0000","last_active":"2024-04-28T12:37:27.838968+0000","last_peered":"2024-04-28T12:37:27.838968+0000","last_clean":"2024-04-28T12:37:27.838968+0000","last_became_active":"2024-04-28T12:37:22.793683+0000","last_became_peered":"2024-04-28T12:37:22.793683+0000","last_unstale":"2024-04-28T12:37:27.838968+0000","last_undegraded":"2024-04-28T12:37:27.838968+0000","last_fullsized":"2024-04-28T12:37:27.838968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:28:33.269962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978735+0000","last_change":"2024-04-28T12:37:22.791966+0000","last_active":"2024-04-28T12:37:27.978735+0000","last_peered":"2024-04-28T12:37:27.978735+0000","last_clean":"2024-04-28T12:37:27.978735+0000","last_became_active":"2024-04-28T12:37:22.791432+0000","last_became_peered":"2024-04-28T12:37:22.791432+0000","last_unstale":"2024-04-28T12:37:27.978735+0000","last_undegraded":"2024-04-28T12:37:27.978735+0000","last_fullsized":"2024-04-28T12:37:27.978735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:47.693882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977566+0000","last_change":"2024-04-28T12:37:22.786169+0000","last_active":"2024-04-28T12:37:27.977566+0000","last_peered":"2024-04-28T12:37:27.977566+0000","last_clean":"2024-04-28T12:37:27.977566+0000","last_became_active":"2024-04-28T12:37:22.785899+0000","last_became_peered":"2024-04-28T12:37:22.785899+0000","last_unstale":"2024-04-28T12:37:27.977566+0000","last_undegraded":"2024-04-28T12:37:27.977566+0000","last_fullsized":"2024-04-28T12:37:27.977566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:06:12.328002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.979102+0000","last_change":"2024-04-28T12:37:22.812114+0000","last_active":"2024-04-28T12:37:27.979102+0000","last_peered":"2024-04-28T12:37:27.979102+0000","last_clean":"2024-04-28T12:37:27.979102+0000","last_became_active":"2024-04-28T12:37:22.808394+0000","last_became_peered":"2024-04-28T12:37:22.808394+0000","last_unstale":"2024-04-28T12:37:27.979102+0000","last_undegraded":"2024-04-28T12:37:27.979102+0000","last_fullsized":"2024-04-28T12:37:27.979102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976044+0000","last_change":"2024-04-28T12:37:22.803864+0000","last_active":"2024-04-28T12:37:27.976044+0000","last_peered":"2024-04-28T12:37:27.976044+0000","last_clean":"2024-04-28T12:37:27.976044+0000","last_became_active":"2024-04-28T12:37:22.803568+0000","last_became_peered":"2024-04-28T12:37:22.803568+0000","last_unstale":"2024-04-28T12:37:27.976044+0000","last_undegraded":"2024-04-28T12:37:27.976044+0000","last_fullsized":"2024-04-28T12:37:27.976044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844461+0000","last_change":"2024-04-28T12:37:22.788814+0000","last_active":"2024-04-28T12:37:27.844461+0000","last_peered":"2024-04-28T12:37:27.844461+0000","last_clean":"2024-04-28T12:37:27.844461+0000","last_became_active":"2024-04-28T12:37:22.786716+0000","last_became_peered":"2024-04-28T12:37:22.786716+0000","last_unstale":"2024-04-28T12:37:27.844461+0000","last_undegraded":"2024-04-28T12:37:27.844461+0000","last_fullsized":"2024-04-28T12:37:27.844461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978543+0000","last_change":"2024-04-28T12:37:22.812645+0000","last_active":"2024-04-28T12:37:27.978543+0000","last_peered":"2024-04-28T12:37:27.978543+0000","last_clean":"2024-04-28T12:37:27.978543+0000","last_became_active":"2024-04-28T12:37:22.811034+0000","last_became_peered":"2024-04-28T12:37:22.811034+0000","last_unstale":"2024-04-28T12:37:27.978543+0000","last_undegraded":"2024-04-28T12:37:27.978543+0000","last_fullsized":"2024-04-28T12:37:27.978543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978281+0000","last_change":"2024-04-28T12:37:22.804060+0000","last_active":"2024-04-28T12:37:27.978281+0000","last_peered":"2024-04-28T12:37:27.978281+0000","last_clean":"2024-04-28T12:37:27.978281+0000","last_became_active":"2024-04-28T12:37:22.803757+0000","last_became_peered":"2024-04-28T12:37:22.803757+0000","last_unstale":"2024-04-28T12:37:27.978281+0000","last_undegraded":"2024-04-28T12:37:27.978281+0000","last_fullsized":"2024-04-28T12:37:27.978281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977634+0000","last_change":"2024-04-28T12:37:22.795301+0000","last_active":"2024-04-28T12:37:27.977634+0000","last_peered":"2024-04-28T12:37:27.977634+0000","last_clean":"2024-04-28T12:37:27.977634+0000","last_became_active":"2024-04-28T12:37:22.794917+0000","last_became_peered":"2024-04-28T12:37:22.794917+0000","last_unstale":"2024-04-28T12:37:27.977634+0000","last_undegraded":"2024-04-28T12:37:27.977634+0000","last_fullsized":"2024-04-28T12:37:27.977634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:45.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978597+0000","last_change":"2024-04-28T12:37:22.805760+0000","last_active":"2024-04-28T12:37:27.978597+0000","last_peered":"2024-04-28T12:37:27.978597+0000","last_clean":"2024-04-28T12:37:27.978597+0000","last_became_active":"2024-04-28T12:37:22.805238+0000","last_became_peered":"2024-04-28T12:37:22.805238+0000","last_unstale":"2024-04-28T12:37:27.978597+0000","last_undegraded":"2024-04-28T12:37:27.978597+0000","last_fullsized":"2024-04-28T12:37:27.978597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:00:16.089959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976930+0000","last_change":"2024-04-28T12:37:22.793207+0000","last_active":"2024-04-28T12:37:27.976930+0000","last_peered":"2024-04-28T12:37:27.976930+0000","last_clean":"2024-04-28T12:37:27.976930+0000","last_became_active":"2024-04-28T12:37:22.792816+0000","last_became_peered":"2024-04-28T12:37:22.792816+0000","last_unstale":"2024-04-28T12:37:27.976930+0000","last_undegraded":"2024-04-28T12:37:27.976930+0000","last_fullsized":"2024-04-28T12:37:27.976930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:55:36.416184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.842925+0000","last_change":"2024-04-28T12:37:22.780772+0000","last_active":"2024-04-28T12:37:27.842925+0000","last_peered":"2024-04-28T12:37:27.842925+0000","last_clean":"2024-04-28T12:37:27.842925+0000","last_became_active":"2024-04-28T12:37:22.780190+0000","last_became_peered":"2024-04-28T12:37:22.780190+0000","last_unstale":"2024-04-28T12:37:27.842925+0000","last_undegraded":"2024-04-28T12:37:27.842925+0000","last_fullsized":"2024-04-28T12:37:27.842925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:45:31.070316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.982761+0000","last_change":"2024-04-28T12:37:22.799792+0000","last_active":"2024-04-28T12:37:27.982761+0000","last_peered":"2024-04-28T12:37:27.982761+0000","last_clean":"2024-04-28T12:37:27.982761+0000","last_became_active":"2024-04-28T12:37:22.799472+0000","last_became_peered":"2024-04-28T12:37:22.799472+0000","last_unstale":"2024-04-28T12:37:27.982761+0000","last_undegraded":"2024-04-28T12:37:27.982761+0000","last_fullsized":"2024-04-28T12:37:27.982761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:16:47.693882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.840300+0000","last_change":"2024-04-28T12:37:22.793748+0000","last_active":"2024-04-28T12:37:27.840300+0000","last_peered":"2024-04-28T12:37:27.840300+0000","last_clean":"2024-04-28T12:37:27.840300+0000","last_became_active":"2024-04-28T12:37:22.793461+0000","last_became_peered":"2024-04-28T12:37:22.793461+0000","last_unstale":"2024-04-28T12:37:27.840300+0000","last_undegraded":"2024-04-28T12:37:27.840300+0000","last_fullsized":"2024-04-28T12:37:27.840300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:00:05.608247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980132+0000","last_change":"2024-04-28T12:37:22.816550+0000","last_active":"2024-04-28T12:37:27.980132+0000","last_peered":"2024-04-28T12:37:27.980132+0000","last_clean":"2024-04-28T12:37:27.980132+0000","last_became_active":"2024-04-28T12:37:22.811895+0000","last_became_peered":"2024-04-28T12:37:22.811895+0000","last_unstale":"2024-04-28T12:37:27.980132+0000","last_undegraded":"2024-04-28T12:37:27.980132+0000","last_fullsized":"2024-04-28T12:37:27.980132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:24.827452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977952+0000","last_change":"2024-04-28T12:37:22.803909+0000","last_active":"2024-04-28T12:37:27.977952+0000","last_peered":"2024-04-28T12:37:27.977952+0000","last_clean":"2024-04-28T12:37:27.977952+0000","last_became_active":"2024-04-28T12:37:22.803577+0000","last_became_peered":"2024-04-28T12:37:22.803577+0000","last_unstale":"2024-04-28T12:37:27.977952+0000","last_undegraded":"2024-04-28T12:37:27.977952+0000","last_fullsized":"2024-04-28T12:37:27.977952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:59.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.975883+0000","last_change":"2024-04-28T12:37:22.808602+0000","last_active":"2024-04-28T12:37:27.975883+0000","last_peered":"2024-04-28T12:37:27.975883+0000","last_clean":"2024-04-28T12:37:27.975883+0000","last_became_active":"2024-04-28T12:37:22.808300+0000","last_became_peered":"2024-04-28T12:37:22.808300+0000","last_unstale":"2024-04-28T12:37:27.975883+0000","last_undegraded":"2024-04-28T12:37:27.975883+0000","last_fullsized":"2024-04-28T12:37:27.975883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977024+0000","last_change":"2024-04-28T12:37:22.792556+0000","last_active":"2024-04-28T12:37:27.977024+0000","last_peered":"2024-04-28T12:37:27.977024+0000","last_clean":"2024-04-28T12:37:27.977024+0000","last_became_active":"2024-04-28T12:37:22.789568+0000","last_became_peered":"2024-04-28T12:37:22.789568+0000","last_unstale":"2024-04-28T12:37:27.977024+0000","last_undegraded":"2024-04-28T12:37:27.977024+0000","last_fullsized":"2024-04-28T12:37:27.977024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:12:36.729037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.977547+0000","last_change":"2024-04-28T12:37:22.798420+0000","last_active":"2024-04-28T12:37:27.977547+0000","last_peered":"2024-04-28T12:37:27.977547+0000","last_clean":"2024-04-28T12:37:27.977547+0000","last_became_active":"2024-04-28T12:37:22.798125+0000","last_became_peered":"2024-04-28T12:37:22.798125+0000","last_unstale":"2024-04-28T12:37:27.977547+0000","last_undegraded":"2024-04-28T12:37:27.977547+0000","last_fullsized":"2024-04-28T12:37:27.977547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:42.893320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.845776+0000","last_change":"2024-04-28T12:37:22.799783+0000","last_active":"2024-04-28T12:37:27.845776+0000","last_peered":"2024-04-28T12:37:27.845776+0000","last_clean":"2024-04-28T12:37:27.845776+0000","last_became_active":"2024-04-28T12:37:22.796468+0000","last_became_peered":"2024-04-28T12:37:22.796468+0000","last_unstale":"2024-04-28T12:37:27.845776+0000","last_undegraded":"2024-04-28T12:37:27.845776+0000","last_fullsized":"2024-04-28T12:37:27.845776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:00.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844178+0000","last_change":"2024-04-28T12:37:22.806029+0000","last_active":"2024-04-28T12:37:27.844178+0000","last_peered":"2024-04-28T12:37:27.844178+0000","last_clean":"2024-04-28T12:37:27.844178+0000","last_became_active":"2024-04-28T12:37:22.802600+0000","last_became_peered":"2024-04-28T12:37:22.802600+0000","last_unstale":"2024-04-28T12:37:27.844178+0000","last_undegraded":"2024-04-28T12:37:27.844178+0000","last_fullsized":"2024-04-28T12:37:27.844178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:10.503634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.844198+0000","last_change":"2024-04-28T12:37:22.821112+0000","last_active":"2024-04-28T12:37:27.844198+0000","last_peered":"2024-04-28T12:37:27.844198+0000","last_clean":"2024-04-28T12:37:27.844198+0000","last_became_active":"2024-04-28T12:37:22.820777+0000","last_became_peered":"2024-04-28T12:37:22.820777+0000","last_unstale":"2024-04-28T12:37:27.844198+0000","last_undegraded":"2024-04-28T12:37:27.844198+0000","last_fullsized":"2024-04-28T12:37:27.844198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981775+0000","last_change":"2024-04-28T12:37:22.825673+0000","last_active":"2024-04-28T12:37:27.981775+0000","last_peered":"2024-04-28T12:37:27.981775+0000","last_clean":"2024-04-28T12:37:27.981775+0000","last_became_active":"2024-04-28T12:37:22.820991+0000","last_became_peered":"2024-04-28T12:37:22.820991+0000","last_unstale":"2024-04-28T12:37:27.981775+0000","last_undegraded":"2024-04-28T12:37:27.981775+0000","last_fullsized":"2024-04-28T12:37:27.981775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:07:00.502362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976734+0000","last_change":"2024-04-28T12:37:22.814076+0000","last_active":"2024-04-28T12:37:27.976734+0000","last_peered":"2024-04-28T12:37:27.976734+0000","last_clean":"2024-04-28T12:37:27.976734+0000","last_became_active":"2024-04-28T12:37:22.813739+0000","last_became_peered":"2024-04-28T12:37:22.813739+0000","last_unstale":"2024-04-28T12:37:27.976734+0000","last_undegraded":"2024-04-28T12:37:27.976734+0000","last_fullsized":"2024-04-28T12:37:27.976734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:04:24.741606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.980558+0000","last_change":"2024-04-28T12:37:22.799783+0000","last_active":"2024-04-28T12:37:27.980558+0000","last_peered":"2024-04-28T12:37:27.980558+0000","last_clean":"2024-04-28T12:37:27.980558+0000","last_became_active":"2024-04-28T12:37:22.799519+0000","last_became_peered":"2024-04-28T12:37:22.799519+0000","last_unstale":"2024-04-28T12:37:27.980558+0000","last_undegraded":"2024-04-28T12:37:27.980558+0000","last_fullsized":"2024-04-28T12:37:27.980558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:59.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978456+0000","last_change":"2024-04-28T12:37:22.805539+0000","last_active":"2024-04-28T12:37:27.978456+0000","last_peered":"2024-04-28T12:37:27.978456+0000","last_clean":"2024-04-28T12:37:27.978456+0000","last_became_active":"2024-04-28T12:37:22.804834+0000","last_became_peered":"2024-04-28T12:37:22.804834+0000","last_unstale":"2024-04-28T12:37:27.978456+0000","last_undegraded":"2024-04-28T12:37:27.978456+0000","last_fullsized":"2024-04-28T12:37:27.978456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:34:47.618755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981101+0000","last_change":"2024-04-28T12:37:22.800776+0000","last_active":"2024-04-28T12:37:27.981101+0000","last_peered":"2024-04-28T12:37:27.981101+0000","last_clean":"2024-04-28T12:37:27.981101+0000","last_became_active":"2024-04-28T12:37:22.798852+0000","last_became_peered":"2024-04-28T12:37:22.798852+0000","last_unstale":"2024-04-28T12:37:27.981101+0000","last_undegraded":"2024-04-28T12:37:27.981101+0000","last_fullsized":"2024-04-28T12:37:27.981101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:30:24.827452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978903+0000","last_change":"2024-04-28T12:37:22.819772+0000","last_active":"2024-04-28T12:37:27.978903+0000","last_peered":"2024-04-28T12:37:27.978903+0000","last_clean":"2024-04-28T12:37:27.978903+0000","last_became_active":"2024-04-28T12:37:22.813375+0000","last_became_peered":"2024-04-28T12:37:22.813375+0000","last_unstale":"2024-04-28T12:37:27.978903+0000","last_undegraded":"2024-04-28T12:37:27.978903+0000","last_fullsized":"2024-04-28T12:37:27.978903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:13.677736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.978010+0000","last_change":"2024-04-28T12:37:22.784463+0000","last_active":"2024-04-28T12:37:27.978010+0000","last_peered":"2024-04-28T12:37:27.978010+0000","last_clean":"2024-04-28T12:37:27.978010+0000","last_became_active":"2024-04-28T12:37:22.784183+0000","last_became_peered":"2024-04-28T12:37:22.784183+0000","last_unstale":"2024-04-28T12:37:27.978010+0000","last_undegraded":"2024-04-28T12:37:27.978010+0000","last_fullsized":"2024-04-28T12:37:27.978010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:51:23.508261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.976443+0000","last_change":"2024-04-28T12:37:22.802869+0000","last_active":"2024-04-28T12:37:27.976443+0000","last_peered":"2024-04-28T12:37:27.976443+0000","last_clean":"2024-04-28T12:37:27.976443+0000","last_became_active":"2024-04-28T12:37:22.802598+0000","last_became_peered":"2024-04-28T12:37:22.802598+0000","last_unstale":"2024-04-28T12:37:27.976443+0000","last_undegraded":"2024-04-28T12:37:27.976443+0000","last_fullsized":"2024-04-28T12:37:27.976443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:55:46.777933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-28T12:37:27.981568+0000","last_change":"2024-04-28T12:37:22.797900+0000","last_active":"2024-04-28T12:37:27.981568+0000","last_peered":"2024-04-28T12:37:27.981568+0000","last_clean":"2024-04-28T12:37:27.981568+0000","last_became_active":"2024-04-28T12:37:22.797533+0000","last_became_peered":"2024-04-28T12:37:22.797533+0000","last_unstale":"2024-04-28T12:37:27.981568+0000","last_undegraded":"2024-04-28T12:37:27.981568+0000","last_fullsized":"2024-04-28T12:37:27.981568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:21.749115+0000","last_clean_scrub_stamp":"2024-04-28T12:37:21.749115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:08:46.974988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:37:56.646 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-28T12:37:56.646 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-28T12:37:56.646 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-28T12:37:56.646 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-28T12:37:56.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.881+0000 7f96eae00640 1 Processor -- start 2024-04-28T12:37:56.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.882+0000 7f96eae00640 1 -- start start 2024-04-28T12:37:56.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.883+0000 7f96eae00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f96e415cc00 0x7f96e406ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.883+0000 7f96eae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e406e410 0x7f96e415f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.883+0000 7f96eae00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f96e415fc00 0x7f96e4163300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.883+0000 7f96eae00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f96e40d1850 con 0x7f96e415fc00 2024-04-28T12:37:56.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.883+0000 7f96eae00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f96e4073e70 con 0x7f96e415cc00 2024-04-28T12:37:56.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.883+0000 7f96eae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96e40d0fb0 con 0x7f96e406e410 2024-04-28T12:37:56.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.884+0000 7f96e8e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f96e415fc00 0x7f96e4163300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:56.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.884+0000 7f96e8e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f96e415fc00 0x7f96e4163300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45456/0 (socket says 172.21.15.131:45456) 2024-04-28T12:37:56.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.884+0000 7f96e8e00640 1 -- 172.21.15.131:0/1602860387 learned_addr learned my addr 172.21.15.131:0/1602860387 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:56.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.884+0000 7f96db400640 1 --2- 172.21.15.131:0/1602860387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e406e410 0x7f96e415f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:56.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.884+0000 7f96e3e00640 1 --2- 172.21.15.131:0/1602860387 >> v2:172.21.15.157:3300/0 conn(0x7f96e415cc00 0x7f96e406ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:56.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.884+0000 7f96e8e00640 1 -- 172.21.15.131:0/1602860387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e406e410 msgr2=0x7f96e415f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:56.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.884+0000 7f96e8e00640 1 --2- 172.21.15.131:0/1602860387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e406e410 0x7f96e415f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.884+0000 7f96e8e00640 1 -- 172.21.15.131:0/1602860387 >> v2:172.21.15.157:3300/0 conn(0x7f96e415cc00 msgr2=0x7f96e406ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:56.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.884+0000 7f96e8e00640 1 --2- 172.21.15.131:0/1602860387 >> v2:172.21.15.157:3300/0 conn(0x7f96e415cc00 0x7f96e406ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.884+0000 7f96e8e00640 1 -- 172.21.15.131:0/1602860387 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e40d1290 con 0x7f96e415fc00 2024-04-28T12:37:56.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.884+0000 7f96e3e00640 1 --2- 172.21.15.131:0/1602860387 >> v2:172.21.15.157:3300/0 conn(0x7f96e415cc00 0x7f96e406ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:56.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.885+0000 7f96e8e00640 1 --2- 172.21.15.131:0/1602860387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e415fc00 0x7f96e4163300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f96e40d1ab0 tx=0x7f96dc02a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763e2bc3fe60793 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:56.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.885+0000 7f96e3400640 1 -- 172.21.15.131:0/1602860387 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc036070 con 0x7f96e415fc00 2024-04-28T12:37:56.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.885+0000 7f96e3400640 1 -- 172.21.15.131:0/1602860387 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96dc031600 con 0x7f96e415fc00 2024-04-28T12:37:56.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.885+0000 7f96e3400640 1 -- 172.21.15.131:0/1602860387 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0319d0 con 0x7f96e415fc00 2024-04-28T12:37:56.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.886+0000 7f96eae00640 1 -- 172.21.15.131:0/1602860387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e415fc00 msgr2=0x7f96e4163300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:56.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.886+0000 7f96eae00640 1 --2- 172.21.15.131:0/1602860387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e415fc00 0x7f96e4163300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f96e40d1ab0 tx=0x7f96dc02a600 comp rx=0 tx=0).stop 2024-04-28T12:37:56.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.886+0000 7f96eae00640 1 -- 172.21.15.131:0/1602860387 shutdown_connections 2024-04-28T12:37:56.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.886+0000 7f96eae00640 1 --2- 172.21.15.131:0/1602860387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e415fc00 0x7f96e4163300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.886+0000 7f96eae00640 1 --2- 172.21.15.131:0/1602860387 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e406e410 0x7f96e415f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.886+0000 7f96eae00640 1 --2- 172.21.15.131:0/1602860387 >> v2:172.21.15.157:3300/0 conn(0x7f96e415cc00 0x7f96e406ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.886+0000 7f96eae00640 1 -- 172.21.15.131:0/1602860387 >> 172.21.15.131:0/1602860387 conn(0x7f96e40682a0 msgr2=0x7f96e40686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:56.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.886+0000 7f96eae00640 1 -- 172.21.15.131:0/1602860387 shutdown_connections 2024-04-28T12:37:56.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.886+0000 7f96eae00640 1 -- 172.21.15.131:0/1602860387 wait complete. 2024-04-28T12:37:56.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.888+0000 7f96eae00640 1 Processor -- start 2024-04-28T12:37:56.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.888+0000 7f96eae00640 1 -- start start 2024-04-28T12:37:56.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96eae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e406e410 0x7f96e4161b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96eae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e415cc00 0x7f96e41620c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96eae00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e415fc00 0x7f96e41f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96eae00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f96e415cfe0 con 0x7f96e415fc00 2024-04-28T12:37:56.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96eae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96e4162de0 con 0x7f96e415cc00 2024-04-28T12:37:56.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96eae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96e415d160 con 0x7f96e406e410 2024-04-28T12:37:56.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96e8e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e415fc00 0x7f96e41f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:56.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96e8e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e415fc00 0x7f96e41f4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45472/0 (socket says 172.21.15.131:45472) 2024-04-28T12:37:56.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96e8e00640 1 -- 172.21.15.131:0/3905651742 learned_addr learned my addr 172.21.15.131:0/3905651742 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:56.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96e3e00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e406e410 0x7f96e4161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:56.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96db400640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e415cc00 0x7f96e41620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:56.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96e8e00640 1 -- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e406e410 msgr2=0x7f96e4161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:56.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96e8e00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e406e410 0x7f96e4161b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96e8e00640 1 -- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e415cc00 msgr2=0x7f96e41620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:56.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96e8e00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e415cc00 0x7f96e41620c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:56.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96e8e00640 1 -- 172.21.15.131:0/3905651742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e41f61b0 con 0x7f96e415fc00 2024-04-28T12:37:56.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.889+0000 7f96e3e00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e406e410 0x7f96e4161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:56.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.890+0000 7f96e8e00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e415fc00 0x7f96e41f4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f96dc02abf0 tx=0x7f96dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:56.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.890+0000 7f96e0c00640 1 -- 172.21.15.131:0/3905651742 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc03f020 con 0x7f96e415fc00 2024-04-28T12:37:56.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.890+0000 7f96e0c00640 1 -- 172.21.15.131:0/3905651742 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96dc037040 con 0x7f96e415fc00 2024-04-28T12:37:56.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.890+0000 7f96e0c00640 1 -- 172.21.15.131:0/3905651742 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc036070 con 0x7f96e415fc00 2024-04-28T12:37:56.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.890+0000 7f96eae00640 1 -- 172.21.15.131:0/3905651742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e41f6490 con 0x7f96e415fc00 2024-04-28T12:37:56.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.890+0000 7f96eae00640 1 -- 172.21.15.131:0/3905651742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e41f6a50 con 0x7f96e415fc00 2024-04-28T12:37:56.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.891+0000 7f96eae00640 1 -- 172.21.15.131:0/3905651742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e4069350 con 0x7f96e415fc00 2024-04-28T12:37:56.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.892+0000 7f96e0c00640 1 -- 172.21.15.131:0/3905651742 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96dc042020 con 0x7f96e415fc00 2024-04-28T12:37:56.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.893+0000 7f96e0c00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f96c006c920 0x7f96c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:56.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.893+0000 7f96e0c00640 1 -- 172.21.15.131:0/3905651742 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f96dc0ac3f0 con 0x7f96e415fc00 2024-04-28T12:37:56.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.893+0000 7f96e3e00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f96c006c920 0x7f96c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:56.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.895+0000 7f96e3e00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f96c006c920 0x7f96c006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006fd0 tx=0x7f96d4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:56.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:56.898+0000 7f96e0c00640 1 -- 172.21.15.131:0/3905651742 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96dc049030 con 0x7f96e415fc00 2024-04-28T12:37:57.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.124+0000 7f96eae00640 1 -- 172.21.15.131:0/3905651742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f96e4072290 con 0x7f96e415fc00 2024-04-28T12:37:57.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.125+0000 7f96e0c00640 1 -- 172.21.15.131:0/3905651742 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f96dc02ae20 con 0x7f96e415fc00 2024-04-28T12:37:57.122 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:57.122 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-28T12:37:57.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.128+0000 7f96eae00640 1 -- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f96c006c920 msgr2=0x7f96c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.128+0000 7f96eae00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f96c006c920 0x7f96c006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006fd0 tx=0x7f96d4008040 comp rx=0 tx=0).stop 2024-04-28T12:37:57.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.128+0000 7f96eae00640 1 -- 172.21.15.131:0/3905651742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e415fc00 msgr2=0x7f96e41f4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.128+0000 7f96eae00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e415fc00 0x7f96e41f4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f96dc02abf0 tx=0x7f96dc004a40 comp rx=0 tx=0).stop 2024-04-28T12:37:57.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.128+0000 7f96eae00640 1 -- 172.21.15.131:0/3905651742 shutdown_connections 2024-04-28T12:37:57.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.128+0000 7f96eae00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f96c006c920 0x7f96c006ede0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.128+0000 7f96eae00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96e415fc00 0x7f96e41f4e50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.128+0000 7f96eae00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96e415cc00 0x7f96e41620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.128+0000 7f96eae00640 1 --2- 172.21.15.131:0/3905651742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96e406e410 0x7f96e4161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.128+0000 7f96eae00640 1 -- 172.21.15.131:0/3905651742 >> 172.21.15.131:0/3905651742 conn(0x7f96e40682a0 msgr2=0x7f96e40ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:57.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.128+0000 7f96eae00640 1 -- 172.21.15.131:0/3905651742 shutdown_connections 2024-04-28T12:37:57.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.128+0000 7f96eae00640 1 -- 172.21.15.131:0/3905651742 wait complete. 2024-04-28T12:37:57.144 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-28T12:37:57.144 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:37:57.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.342+0000 7f6606c00640 1 Processor -- start 2024-04-28T12:37:57.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.342+0000 7f6606c00640 1 -- start start 2024-04-28T12:37:57.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.343+0000 7f6606c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f66000cd180 0x7f6600072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:57.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.343+0000 7f6606c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66000cd560 con 0x7f660015dc50 2024-04-28T12:37:57.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.343+0000 7f6606c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6600072d00 con 0x7f66000ce480 2024-04-28T12:37:57.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.343+0000 7f6606c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6600072fe0 con 0x7f66000cd180 2024-04-28T12:37:57.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.343+0000 7f6605800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f66000cd180 0x7f6600072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:57.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.343+0000 7f6604e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f660015dc50 0x7f66000cdd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33252/0 (socket says 172.21.15.131:33252) 2024-04-28T12:37:57.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.343+0000 7f6604e00640 1 -- 172.21.15.131:0/3435108736 learned_addr learned my addr 172.21.15.131:0/3435108736 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.344+0000 7f6605800640 1 -- 172.21.15.131:0/3435108736 >> v1:172.21.15.157:6789/0 conn(0x7f66000ce480 legacy=0x7f660006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.344+0000 7f6605800640 1 -- 172.21.15.131:0/3435108736 >> v1:172.21.15.131:6789/0 conn(0x7f660015dc50 legacy=0x7f66000cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.344+0000 7f6605800640 1 -- 172.21.15.131:0/3435108736 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66000732c0 con 0x7f66000cd180 2024-04-28T12:37:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.344+0000 7f65ffe00640 1 -- 172.21.15.131:0/3435108736 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2317934554 0 0) 0x7f66000cd560 con 0x7f660015dc50 2024-04-28T12:37:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.345+0000 7f6605800640 1 --2- 172.21.15.131:0/3435108736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66000cd180 0x7f6600072620 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009d90 tx=0x7f65f402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9028d3ac905d59d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.346+0000 7f65ffe00640 1 -- 172.21.15.131:0/3435108736 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4007740 con 0x7f66000cd180 2024-04-28T12:37:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.346+0000 7f65ffe00640 1 -- 172.21.15.131:0/3435108736 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f40078e0 con 0x7f66000cd180 2024-04-28T12:37:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.346+0000 7f65ffe00640 1 -- 172.21.15.131:0/3435108736 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4007cb0 con 0x7f66000cd180 2024-04-28T12:37:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.346+0000 7f6606c00640 1 -- 172.21.15.131:0/3435108736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66000cd180 msgr2=0x7f6600072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.346+0000 7f6606c00640 1 --2- 172.21.15.131:0/3435108736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66000cd180 0x7f6600072620 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009d90 tx=0x7f65f402a6a0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.346+0000 7f6606c00640 1 -- 172.21.15.131:0/3435108736 shutdown_connections 2024-04-28T12:37:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.346+0000 7f6606c00640 1 --2- 172.21.15.131:0/3435108736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66000cd180 0x7f6600072620 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.346+0000 7f6606c00640 1 -- 172.21.15.131:0/3435108736 >> 172.21.15.131:0/3435108736 conn(0x7f6600068090 msgr2=0x7f6600068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.346+0000 7f6606c00640 1 -- 172.21.15.131:0/3435108736 shutdown_connections 2024-04-28T12:37:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.346+0000 7f6606c00640 1 -- 172.21.15.131:0/3435108736 wait complete. 2024-04-28T12:37:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.348+0000 7f6606c00640 1 Processor -- start 2024-04-28T12:37:57.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.348+0000 7f6606c00640 1 -- start start 2024-04-28T12:37:57.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.349+0000 7f6606c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000cd180 0x7f66001e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:57.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.349+0000 7f6606c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66000ce480 0x7f66000af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:57.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.349+0000 7f6606c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f660015dc50 0x7f660015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:57.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.349+0000 7f6606c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f660015e0d0 con 0x7f660015dc50 2024-04-28T12:37:57.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.349+0000 7f6606c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6600073da0 con 0x7f66000ce480 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.349+0000 7f6606c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f660006e1d0 con 0x7f66000cd180 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.349+0000 7f6606200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f660015dc50 0x7f660015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.350+0000 7f6606200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f660015dc50 0x7f660015c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45482/0 (socket says 172.21.15.131:45482) 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.350+0000 7f6606200640 1 -- 172.21.15.131:0/4237924488 learned_addr learned my addr 172.21.15.131:0/4237924488 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.350+0000 7f6604e00640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66000ce480 0x7f66000af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.350+0000 7f6605800640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000cd180 0x7f66001e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.350+0000 7f6606200640 1 -- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000cd180 msgr2=0x7f66001e76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.350+0000 7f6606200640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000cd180 0x7f66001e76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.350+0000 7f6606200640 1 -- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66000ce480 msgr2=0x7f66000af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.350+0000 7f6606200640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66000ce480 0x7f66000af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.350+0000 7f6606200640 1 -- 172.21.15.131:0/4237924488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660015c790 con 0x7f660015dc50 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.350+0000 7f6605800640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000cd180 0x7f66001e76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.350+0000 7f6606200640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f660015dc50 0x7f660015c250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f65f800c410 tx=0x7f65f800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.350+0000 7f65fd600640 1 -- 172.21.15.131:0/4237924488 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8011790 con 0x7f660015dc50 2024-04-28T12:37:57.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.351+0000 7f65fd600640 1 -- 172.21.15.131:0/4237924488 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f8017070 con 0x7f660015dc50 2024-04-28T12:37:57.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.351+0000 7f65fd600640 1 -- 172.21.15.131:0/4237924488 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8011bb0 con 0x7f660015dc50 2024-04-28T12:37:57.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.351+0000 7f6606c00640 1 -- 172.21.15.131:0/4237924488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66001e7db0 con 0x7f660015dc50 2024-04-28T12:37:57.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.351+0000 7f6606c00640 1 -- 172.21.15.131:0/4237924488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66001e8370 con 0x7f660015dc50 2024-04-28T12:37:57.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.352+0000 7f6606c00640 1 -- 172.21.15.131:0/4237924488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6600069350 con 0x7f660015dc50 2024-04-28T12:37:57.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.353+0000 7f65fd600640 1 -- 172.21.15.131:0/4237924488 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65f801c460 con 0x7f660015dc50 2024-04-28T12:37:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.354+0000 7f65fd600640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f65e006c8f0 0x7f65e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:57.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.354+0000 7f65fd600640 1 -- 172.21.15.131:0/4237924488 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f65f808ba60 con 0x7f660015dc50 2024-04-28T12:37:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.354+0000 7f6605800640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f65e006c8f0 0x7f65e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.355+0000 7f6605800640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f65e006c8f0 0x7f65e006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65f4034000 tx=0x7f65f402f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:57.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.358+0000 7f65fd600640 1 -- 172.21.15.131:0/4237924488 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f65f8021020 con 0x7f660015dc50 2024-04-28T12:37:57.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.556+0000 7f6606c00640 1 -- 172.21.15.131:0/4237924488 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66000d0670 con 0x7f660015dc50 2024-04-28T12:37:57.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.559+0000 7f65fd600640 1 -- 172.21.15.131:0/4237924488 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7f65f8059b50 con 0x7f660015dc50 2024-04-28T12:37:57.556 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:57.556 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:37:57.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.562+0000 7f6606c00640 1 -- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f65e006c8f0 msgr2=0x7f65e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.562+0000 7f6606c00640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f65e006c8f0 0x7f65e006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65f4034000 tx=0x7f65f402f000 comp rx=0 tx=0).stop 2024-04-28T12:37:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.562+0000 7f6606c00640 1 -- 172.21.15.131:0/4237924488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f660015dc50 msgr2=0x7f660015c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.562+0000 7f6606c00640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f660015dc50 0x7f660015c250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f65f800c410 tx=0x7f65f800c8d0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.562+0000 7f6606c00640 1 -- 172.21.15.131:0/4237924488 shutdown_connections 2024-04-28T12:37:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.562+0000 7f6606c00640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f65e006c8f0 0x7f65e006edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.562+0000 7f6606c00640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f660015dc50 0x7f660015c250 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.562+0000 7f6606c00640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66000ce480 0x7f66000af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.562+0000 7f6606c00640 1 --2- 172.21.15.131:0/4237924488 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66000cd180 0x7f66001e76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.562+0000 7f6606c00640 1 -- 172.21.15.131:0/4237924488 >> 172.21.15.131:0/4237924488 conn(0x7f6600068090 msgr2=0x7f66000cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.562+0000 7f6606c00640 1 -- 172.21.15.131:0/4237924488 shutdown_connections 2024-04-28T12:37:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.562+0000 7f6606c00640 1 -- 172.21.15.131:0/4237924488 wait complete. 2024-04-28T12:37:57.578 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:37:57.578 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:37:57.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.777+0000 7f6f3b200640 1 Processor -- start 2024-04-28T12:37:57.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.778+0000 7f6f3b200640 1 -- start start 2024-04-28T12:37:57.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.779+0000 7f6f3b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f3c160080 0x7f6f3c15c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:57.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.779+0000 7f6f3b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6f3c15f700 0x7f6f3c06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:57.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.779+0000 7f6f3b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6f3c070480 0x7f6f3c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.779+0000 7f6f3b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6f3c06fab0 con 0x7f6f3c070480 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.779+0000 7f6f3b200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6f3c15c960 con 0x7f6f3c15f700 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.779+0000 7f6f3b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f3c15cc40 con 0x7f6f3c160080 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.779+0000 7f6f3a800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6f3c070480 0x7f6f3c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.779+0000 7f6f3a800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6f3c070480 0x7f6f3c15d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45492/0 (socket says 172.21.15.131:45492) 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.779+0000 7f6f3a800640 1 -- 172.21.15.131:0/3974087482 learned_addr learned my addr 172.21.15.131:0/3974087482 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.780+0000 7f6f33e00640 1 --2- 172.21.15.131:0/3974087482 >> v2:172.21.15.157:3300/0 conn(0x7f6f3c15f700 0x7f6f3c06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.780+0000 7f6f39e00640 1 --2- 172.21.15.131:0/3974087482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f3c160080 0x7f6f3c15c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.780+0000 7f6f3a800640 1 -- 172.21.15.131:0/3974087482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f3c160080 msgr2=0x7f6f3c15c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.780+0000 7f6f3a800640 1 --2- 172.21.15.131:0/3974087482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f3c160080 0x7f6f3c15c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.780+0000 7f6f3a800640 1 -- 172.21.15.131:0/3974087482 >> v2:172.21.15.157:3300/0 conn(0x7f6f3c15f700 msgr2=0x7f6f3c06ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.780+0000 7f6f3a800640 1 --2- 172.21.15.131:0/3974087482 >> v2:172.21.15.157:3300/0 conn(0x7f6f3c15f700 0x7f6f3c06ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.780+0000 7f6f3a800640 1 -- 172.21.15.131:0/3974087482 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c15cf20 con 0x7f6f3c070480 2024-04-28T12:37:57.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.781+0000 7f6f3a800640 1 --2- 172.21.15.131:0/3974087482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f3c070480 0x7f6f3c15d2d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c069010 tx=0x7f6f3402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01c21b18d62f7c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:57.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.781+0000 7f6f39400640 1 -- 172.21.15.131:0/3974087482 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34036070 con 0x7f6f3c070480 2024-04-28T12:37:57.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.781+0000 7f6f39400640 1 -- 172.21.15.131:0/3974087482 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f34031600 con 0x7f6f3c070480 2024-04-28T12:37:57.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.781+0000 7f6f39400640 1 -- 172.21.15.131:0/3974087482 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f340319d0 con 0x7f6f3c070480 2024-04-28T12:37:57.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.782+0000 7f6f3b200640 1 -- 172.21.15.131:0/3974087482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f3c070480 msgr2=0x7f6f3c15d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.782+0000 7f6f3b200640 1 --2- 172.21.15.131:0/3974087482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f3c070480 0x7f6f3c15d2d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c069010 tx=0x7f6f3402a610 comp rx=0 tx=0).stop 2024-04-28T12:37:57.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.782+0000 7f6f3b200640 1 -- 172.21.15.131:0/3974087482 shutdown_connections 2024-04-28T12:37:57.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.782+0000 7f6f3b200640 1 --2- 172.21.15.131:0/3974087482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f3c070480 0x7f6f3c15d2d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.782+0000 7f6f3b200640 1 --2- 172.21.15.131:0/3974087482 >> v2:172.21.15.157:3300/0 conn(0x7f6f3c15f700 0x7f6f3c06ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.782+0000 7f6f3b200640 1 --2- 172.21.15.131:0/3974087482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f3c160080 0x7f6f3c15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.782+0000 7f6f3b200640 1 -- 172.21.15.131:0/3974087482 >> 172.21.15.131:0/3974087482 conn(0x7f6f3c068090 msgr2=0x7f6f3c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:57.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.782+0000 7f6f3b200640 1 -- 172.21.15.131:0/3974087482 shutdown_connections 2024-04-28T12:37:57.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.782+0000 7f6f3b200640 1 -- 172.21.15.131:0/3974087482 wait complete. 2024-04-28T12:37:57.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.783+0000 7f6f3b200640 1 Processor -- start 2024-04-28T12:37:57.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.783+0000 7f6f3b200640 1 -- start start 2024-04-28T12:37:57.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.784+0000 7f6f3b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f3c070480 0x7f6f3c14a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:57.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.784+0000 7f6f3b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f3c15f700 0x7f6f3c14b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:57.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.784+0000 7f6f3b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f3c160080 0x7f6f3c14b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:57.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.784+0000 7f6f3b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1605e0 con 0x7f6f3c160080 2024-04-28T12:37:57.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.784+0000 7f6f3b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c160460 con 0x7f6f3c15f700 2024-04-28T12:37:57.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.784+0000 7f6f3b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f3c15d810 con 0x7f6f3c070480 2024-04-28T12:37:57.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f3a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f3c160080 0x7f6f3c14b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:57.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f3a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f3c160080 0x7f6f3c14b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45500/0 (socket says 172.21.15.131:45500) 2024-04-28T12:37:57.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f3a800640 1 -- 172.21.15.131:0/59947477 learned_addr learned my addr 172.21.15.131:0/59947477 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:57.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f39e00640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f3c070480 0x7f6f3c14a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:57.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f33e00640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f3c15f700 0x7f6f3c14b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f3a800640 1 -- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f3c070480 msgr2=0x7f6f3c14a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f3a800640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f3c070480 0x7f6f3c14a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f3a800640 1 -- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f3c15f700 msgr2=0x7f6f3c14b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f3a800640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f3c15f700 0x7f6f3c14b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f3a800640 1 -- 172.21.15.131:0/59947477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c15a850 con 0x7f6f3c160080 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f39e00640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f3c070480 0x7f6f3c14a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f33e00640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f3c15f700 0x7f6f3c14b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.785+0000 7f6f3a800640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f3c160080 0x7f6f3c14b5c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6f3402ac00 tx=0x7f6f34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.786+0000 7f6f32000640 1 -- 172.21.15.131:0/59947477 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3403f020 con 0x7f6f3c160080 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.786+0000 7f6f32000640 1 -- 172.21.15.131:0/59947477 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f34037040 con 0x7f6f3c160080 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.786+0000 7f6f32000640 1 -- 172.21.15.131:0/59947477 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34036070 con 0x7f6f3c160080 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.786+0000 7f6f3b200640 1 -- 172.21.15.131:0/59947477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3c15ab30 con 0x7f6f3c160080 2024-04-28T12:37:57.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.786+0000 7f6f3b200640 1 -- 172.21.15.131:0/59947477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3c15b0f0 con 0x7f6f3c160080 2024-04-28T12:37:57.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.787+0000 7f6f3b200640 1 -- 172.21.15.131:0/59947477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f3c0506c0 con 0x7f6f3c160080 2024-04-28T12:37:57.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.788+0000 7f6f32000640 1 -- 172.21.15.131:0/59947477 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f34042020 con 0x7f6f3c160080 2024-04-28T12:37:57.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.789+0000 7f6f32000640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6f1806c8f0 0x7f6f1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:57.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.789+0000 7f6f32000640 1 -- 172.21.15.131:0/59947477 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6f340ac790 con 0x7f6f3c160080 2024-04-28T12:37:57.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.789+0000 7f6f39e00640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6f1806c8f0 0x7f6f1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:57.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.790+0000 7f6f39e00640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6f1806c8f0 0x7f6f1806edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00b800 tx=0x7f6f2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:57.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.793+0000 7f6f32000640 1 -- 172.21.15.131:0/59947477 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6f34049020 con 0x7f6f3c160080 2024-04-28T12:37:57.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.988+0000 7f6f3b200640 1 -- 172.21.15.131:0/59947477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f3c14ae60 con 0x7f6f3c160080 2024-04-28T12:37:57.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.988+0000 7f6f32000640 1 -- 172.21.15.131:0/59947477 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f6f3407a870 con 0x7f6f3c160080 2024-04-28T12:37:57.985 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:37:57.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.990+0000 7f6f3b200640 1 -- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6f1806c8f0 msgr2=0x7f6f1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.990+0000 7f6f3b200640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6f1806c8f0 0x7f6f1806edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00b800 tx=0x7f6f2c008040 comp rx=0 tx=0).stop 2024-04-28T12:37:57.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.990+0000 7f6f3b200640 1 -- 172.21.15.131:0/59947477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f3c160080 msgr2=0x7f6f3c14b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:57.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.990+0000 7f6f3b200640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f3c160080 0x7f6f3c14b5c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6f3402ac00 tx=0x7f6f34004a40 comp rx=0 tx=0).stop 2024-04-28T12:37:57.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.991+0000 7f6f3b200640 1 -- 172.21.15.131:0/59947477 shutdown_connections 2024-04-28T12:37:57.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.991+0000 7f6f3b200640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6f1806c8f0 0x7f6f1806edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.991+0000 7f6f3b200640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f3c160080 0x7f6f3c14b5c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.991+0000 7f6f3b200640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f3c15f700 0x7f6f3c14b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.991+0000 7f6f3b200640 1 --2- 172.21.15.131:0/59947477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f3c070480 0x7f6f3c14a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:57.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.991+0000 7f6f3b200640 1 -- 172.21.15.131:0/59947477 >> 172.21.15.131:0/59947477 conn(0x7f6f3c068090 msgr2=0x7f6f3c06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:57.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.991+0000 7f6f3b200640 1 -- 172.21.15.131:0/59947477 shutdown_connections 2024-04-28T12:37:57.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:57.991+0000 7f6f3b200640 1 -- 172.21.15.131:0/59947477 wait complete. 2024-04-28T12:37:58.004 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:37:58.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.209+0000 7f4d58e00640 1 Processor -- start 2024-04-28T12:37:58.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.210+0000 7f4d58e00640 1 -- start start 2024-04-28T12:37:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.211+0000 7f4d58e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4d5415ef30 0x7f4d540d5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.211+0000 7f4d58e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d540d3300 0x7f4d54162c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.211+0000 7f4d58e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d540d0d10 con 0x7f4d5415f310 2024-04-28T12:37:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.211+0000 7f4d58e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4d540d3a70 con 0x7f4d5415ef30 2024-04-28T12:37:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.211+0000 7f4d58e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d540cfb20 con 0x7f4d540d3300 2024-04-28T12:37:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.211+0000 7f4d4be00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4d5415f310 0x7f4d540d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33260/0 (socket says 172.21.15.131:33260) 2024-04-28T12:37:58.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d4be00640 1 -- 172.21.15.131:0/707326679 learned_addr learned my addr 172.21.15.131:0/707326679 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:58.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d53400640 1 --2- 172.21.15.131:0/707326679 >> v2:172.21.15.157:3300/0 conn(0x7f4d5415ef30 0x7f4d540d5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d53e00640 1 --2- 172.21.15.131:0/707326679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d540d3300 0x7f4d54162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1731129782 0 0) 0x7f4d540d0d10 con 0x7f4d5415f310 2024-04-28T12:37:58.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d38003520 con 0x7f4d5415f310 2024-04-28T12:37:58.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 909886643 0 0) 0x7f4d38003520 con 0x7f4d5415f310 2024-04-28T12:37:58.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d540d3300 msgr2=0x7f4d54162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:58.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d52a00640 1 --2- 172.21.15.131:0/707326679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d540d3300 0x7f4d54162c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 >> v2:172.21.15.157:3300/0 conn(0x7f4d5415ef30 msgr2=0x7f4d540d5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:58.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d52a00640 1 --2- 172.21.15.131:0/707326679 >> v2:172.21.15.157:3300/0 conn(0x7f4d5415ef30 0x7f4d540d5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d540cfe00 con 0x7f4d5415f310 2024-04-28T12:37:58.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f4d3c001670 con 0x7f4d5415f310 2024-04-28T12:37:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 >> v1:172.21.15.131:6789/0 conn(0x7f4d5415f310 legacy=0x7f4d540d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.212+0000 7f4d52a00640 1 --2- 172.21.15.131:0/707326679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d540d3300 0x7f4d38003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d52a00640 1 --2- 172.21.15.131:0/707326679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d38003fa0 0x7f4d38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d52a00640 1 --2- 172.21.15.131:0/707326679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d38004d40 0x7f4d380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d540d3a70 con 0x7f4d38004d40 2024-04-28T12:37:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d540cfb20 con 0x7f4d38003fa0 2024-04-28T12:37:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d3800ba70 con 0x7f4d540d3300 2024-04-28T12:37:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d53e00640 1 --2- 172.21.15.131:0/707326679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d38004d40 0x7f4d380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d53400640 1 --2- 172.21.15.131:0/707326679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d540d3300 0x7f4d38003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d4be00640 1 --2- 172.21.15.131:0/707326679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d38003fa0 0x7f4d38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d53e00640 1 -- 172.21.15.131:0/707326679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d540d3300 msgr2=0x7f4d38003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:58.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d53e00640 1 --2- 172.21.15.131:0/707326679 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d540d3300 0x7f4d38003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d53e00640 1 -- 172.21.15.131:0/707326679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d38003fa0 msgr2=0x7f4d38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:58.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d53e00640 1 --2- 172.21.15.131:0/707326679 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d38003fa0 0x7f4d38004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.213+0000 7f4d53e00640 1 -- 172.21.15.131:0/707326679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d38003520 con 0x7f4d38004d40 2024-04-28T12:37:58.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.214+0000 7f4d53e00640 1 --2- 172.21.15.131:0/707326679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d38004d40 0x7f4d380051c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009fc0 tx=0x7f4d38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb354668409370c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:58.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.214+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c009c10 con 0x7f4d38004d40 2024-04-28T12:37:58.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.215+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4c036070 con 0x7f4d38004d40 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.215+0000 7f4d52a00640 1 -- 172.21.15.131:0/707326679 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c031880 con 0x7f4d38004d40 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.215+0000 7f4d58e00640 1 -- 172.21.15.131:0/707326679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d38004d40 msgr2=0x7f4d380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.215+0000 7f4d58e00640 1 --2- 172.21.15.131:0/707326679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d38004d40 0x7f4d380051c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009fc0 tx=0x7f4d38005830 comp rx=0 tx=0).stop 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.215+0000 7f4d53e00640 1 -- 172.21.15.131:0/707326679 reap_dead start 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.215+0000 7f4d58e00640 1 -- 172.21.15.131:0/707326679 shutdown_connections 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.215+0000 7f4d58e00640 1 -- 172.21.15.131:0/707326679 >> 172.21.15.131:0/707326679 conn(0x7f4d540d4d20 msgr2=0x7f4d540d5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.215+0000 7f4d58e00640 1 -- 172.21.15.131:0/707326679 shutdown_connections 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.215+0000 7f4d58e00640 1 -- 172.21.15.131:0/707326679 wait complete. 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.217+0000 7f4d58e00640 1 Processor -- start 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.217+0000 7f4d58e00640 1 -- start start 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.218+0000 7f4d58e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d38004d40 0x7f4d54153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.218+0000 7f4d58e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d540b7940 0x7f4d5415b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.218+0000 7f4d58e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d540c8850 0x7f4d540c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.218+0000 7f4d58e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d540cf7f0 con 0x7f4d38004d40 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.218+0000 7f4d58e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d540d3750 con 0x7f4d540b7940 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.218+0000 7f4d58e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d54162c00 con 0x7f4d540c8850 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.219+0000 7f4d53400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d38004d40 0x7f4d54153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.219+0000 7f4d53400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d38004d40 0x7f4d54153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45516/0 (socket says 172.21.15.131:45516) 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.219+0000 7f4d53400640 1 -- 172.21.15.131:0/1298075127 learned_addr learned my addr 172.21.15.131:0/1298075127 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.219+0000 7f4d53400640 1 -- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d540c8850 msgr2=0x7f4d540c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.219+0000 7f4d53400640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d540c8850 0x7f4d540c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.219+0000 7f4d53400640 1 -- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d540b7940 msgr2=0x7f4d5415b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.219+0000 7f4d4be00640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d540b7940 0x7f4d5415b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.219+0000 7f4d53400640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d540b7940 0x7f4d5415b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.219+0000 7f4d53400640 1 -- 172.21.15.131:0/1298075127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d540b7dd0 con 0x7f4d38004d40 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.219+0000 7f4d53400640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d38004d40 0x7f4d54153280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4d44011680 tx=0x7f4d44011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.219+0000 7f4d4b400640 1 -- 172.21.15.131:0/1298075127 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4400f9c0 con 0x7f4d38004d40 2024-04-28T12:37:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.220+0000 7f4d4b400640 1 -- 172.21.15.131:0/1298075127 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4400fb60 con 0x7f4d38004d40 2024-04-28T12:37:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.220+0000 7f4d4b400640 1 -- 172.21.15.131:0/1298075127 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44015460 con 0x7f4d38004d40 2024-04-28T12:37:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.220+0000 7f4d58e00640 1 -- 172.21.15.131:0/1298075127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d540b80b0 con 0x7f4d38004d40 2024-04-28T12:37:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.220+0000 7f4d58e00640 1 -- 172.21.15.131:0/1298075127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d540b8670 con 0x7f4d38004d40 2024-04-28T12:37:58.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.221+0000 7f4d58e00640 1 -- 172.21.15.131:0/1298075127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d540506c0 con 0x7f4d38004d40 2024-04-28T12:37:58.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.222+0000 7f4d4b400640 1 -- 172.21.15.131:0/1298075127 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d44015600 con 0x7f4d38004d40 2024-04-28T12:37:58.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.223+0000 7f4d4b400640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d3006c920 0x7f4d3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.223+0000 7f4d4b400640 1 -- 172.21.15.131:0/1298075127 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4d4408c880 con 0x7f4d38004d40 2024-04-28T12:37:58.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.223+0000 7f4d4be00640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d3006c920 0x7f4d3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.224+0000 7f4d4be00640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d3006c920 0x7f4d3006ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4d5415c180 tx=0x7f4d3c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:58.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.227+0000 7f4d4b400640 1 -- 172.21.15.131:0/1298075127 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d44019030 con 0x7f4d38004d40 2024-04-28T12:37:58.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.426+0000 7f4d58e00640 1 -- 172.21.15.131:0/1298075127 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4d540d2040 con 0x7f4d38004d40 2024-04-28T12:37:58.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.427+0000 7f4d4b400640 1 -- 172.21.15.131:0/1298075127 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f4d4405a8d0 con 0x7f4d38004d40 2024-04-28T12:37:58.423 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:37:58.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.429+0000 7f4d58e00640 1 -- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d3006c920 msgr2=0x7f4d3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:58.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.429+0000 7f4d58e00640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d3006c920 0x7f4d3006ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4d5415c180 tx=0x7f4d3c009040 comp rx=0 tx=0).stop 2024-04-28T12:37:58.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.429+0000 7f4d58e00640 1 -- 172.21.15.131:0/1298075127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d38004d40 msgr2=0x7f4d54153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:58.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.429+0000 7f4d58e00640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d38004d40 0x7f4d54153280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4d44011680 tx=0x7f4d44011b40 comp rx=0 tx=0).stop 2024-04-28T12:37:58.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.429+0000 7f4d58e00640 1 -- 172.21.15.131:0/1298075127 shutdown_connections 2024-04-28T12:37:58.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.429+0000 7f4d58e00640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d3006c920 0x7f4d3006ede0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.429+0000 7f4d58e00640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d540c8850 0x7f4d540c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.429+0000 7f4d58e00640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d540b7940 0x7f4d5415b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.429+0000 7f4d58e00640 1 --2- 172.21.15.131:0/1298075127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d38004d40 0x7f4d54153280 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.429+0000 7f4d58e00640 1 -- 172.21.15.131:0/1298075127 >> 172.21.15.131:0/1298075127 conn(0x7f4d540d4d20 msgr2=0x7f4d540d5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:58.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.430+0000 7f4d58e00640 1 -- 172.21.15.131:0/1298075127 shutdown_connections 2024-04-28T12:37:58.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.430+0000 7f4d58e00640 1 -- 172.21.15.131:0/1298075127 wait complete. 2024-04-28T12:37:58.446 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:37:58.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.651+0000 7f4066a00640 1 Processor -- start 2024-04-28T12:37:58.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.651+0000 7f4066a00640 1 -- start start 2024-04-28T12:37:58.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.653+0000 7f4066a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4060145d80 0x7f4060072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.653+0000 7f4066a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f406015d5e0 0x7f40600728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.653+0000 7f4066a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4060146160 con 0x7f40601468c0 2024-04-28T12:37:58.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.653+0000 7f4066a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f406006dfb0 con 0x7f4060145d80 2024-04-28T12:37:58.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.653+0000 7f4066a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f40600d0470 con 0x7f406015d5e0 2024-04-28T12:37:58.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.653+0000 7f405fe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4060145d80 0x7f4060072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.654+0000 7f405f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f406015d5e0 0x7f40600728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.654+0000 7f405f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f406015d5e0 0x7f40600728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:46804/0 (socket says 172.21.15.131:46804) 2024-04-28T12:37:58.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.654+0000 7f405f400640 1 -- 172.21.15.131:0/3435872779 learned_addr learned my addr 172.21.15.131:0/3435872779 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:58.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.654+0000 7f405f400640 1 -- 172.21.15.131:0/3435872779 >> v2:172.21.15.157:3300/0 conn(0x7f4060145d80 msgr2=0x7f4060072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:58.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.654+0000 7f405f400640 1 --2- 172.21.15.131:0/3435872779 >> v2:172.21.15.157:3300/0 conn(0x7f4060145d80 0x7f4060072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.654+0000 7f405f400640 1 -- 172.21.15.131:0/3435872779 >> v1:172.21.15.157:6789/0 conn(0x7f40601468c0 legacy=0x7f406006ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:58.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.654+0000 7f405f400640 1 -- 172.21.15.131:0/3435872779 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4060074180 con 0x7f406015d5e0 2024-04-28T12:37:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.655+0000 7f405f400640 1 --2- 172.21.15.131:0/3435872779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f406015d5e0 0x7f40600728c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f404c009870 tx=0x7f404c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13126365ec2e42b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.656+0000 7f405ea00640 1 -- 172.21.15.131:0/3435872779 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c037070 con 0x7f406015d5e0 2024-04-28T12:37:58.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.656+0000 7f405ea00640 1 -- 172.21.15.131:0/3435872779 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c032440 con 0x7f406015d5e0 2024-04-28T12:37:58.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.656+0000 7f405ea00640 1 -- 172.21.15.131:0/3435872779 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c032810 con 0x7f406015d5e0 2024-04-28T12:37:58.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.657+0000 7f4066a00640 1 -- 172.21.15.131:0/3435872779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f406015d5e0 msgr2=0x7f40600728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:58.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.657+0000 7f4066a00640 1 --2- 172.21.15.131:0/3435872779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f406015d5e0 0x7f40600728c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f404c009870 tx=0x7f404c02a1c0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.657+0000 7f4066a00640 1 -- 172.21.15.131:0/3435872779 shutdown_connections 2024-04-28T12:37:58.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.657+0000 7f4066a00640 1 --2- 172.21.15.131:0/3435872779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f406015d5e0 0x7f40600728c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.657+0000 7f4066a00640 1 --2- 172.21.15.131:0/3435872779 >> v2:172.21.15.157:3300/0 conn(0x7f4060145d80 0x7f4060072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.657+0000 7f4066a00640 1 -- 172.21.15.131:0/3435872779 >> 172.21.15.131:0/3435872779 conn(0x7f40600b2900 msgr2=0x7f40600b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:58.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.657+0000 7f4066a00640 1 -- 172.21.15.131:0/3435872779 shutdown_connections 2024-04-28T12:37:58.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.657+0000 7f4066a00640 1 -- 172.21.15.131:0/3435872779 wait complete. 2024-04-28T12:37:58.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.659+0000 7f4066a00640 1 Processor -- start 2024-04-28T12:37:58.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.660+0000 7f4066a00640 1 -- start start 2024-04-28T12:37:58.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.661+0000 7f4066a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4060145d80 0x7f40601f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.661+0000 7f4066a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601468c0 0x7f4060143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.661+0000 7f4066a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f406015d5e0 0x7f406013a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.661+0000 7f4066a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f406006e530 con 0x7f40601468c0 2024-04-28T12:37:58.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.661+0000 7f4066a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f406006e250 con 0x7f4060145d80 2024-04-28T12:37:58.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.661+0000 7f4066a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f40600d09f0 con 0x7f406015d5e0 2024-04-28T12:37:58.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.661+0000 7f405f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601468c0 0x7f4060143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.661+0000 7f405f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601468c0 0x7f4060143a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45532/0 (socket says 172.21.15.131:45532) 2024-04-28T12:37:58.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.661+0000 7f405f400640 1 -- 172.21.15.131:0/279549402 learned_addr learned my addr 172.21.15.131:0/279549402 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:58.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.661+0000 7f405fe00640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4060145d80 0x7f40601f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.661+0000 7f4064a00640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f406015d5e0 0x7f406013a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.662+0000 7f405f400640 1 -- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f406015d5e0 msgr2=0x7f406013a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:58.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.662+0000 7f405f400640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f406015d5e0 0x7f406013a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.662+0000 7f405f400640 1 -- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4060145d80 msgr2=0x7f40601f5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:58.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.662+0000 7f405f400640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4060145d80 0x7f40601f5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.662+0000 7f405f400640 1 -- 172.21.15.131:0/279549402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406013bd80 con 0x7f40601468c0 2024-04-28T12:37:58.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.662+0000 7f405f400640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601468c0 0x7f4060143a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f404c000c00 tx=0x7f404c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:58.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.662+0000 7f4057e00640 1 -- 172.21.15.131:0/279549402 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c037040 con 0x7f40601468c0 2024-04-28T12:37:58.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.663+0000 7f4057e00640 1 -- 172.21.15.131:0/279549402 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c042070 con 0x7f40601468c0 2024-04-28T12:37:58.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.663+0000 7f4066a00640 1 -- 172.21.15.131:0/279549402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406013c060 con 0x7f40601468c0 2024-04-28T12:37:58.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.663+0000 7f4057e00640 1 -- 172.21.15.131:0/279549402 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c030880 con 0x7f40601468c0 2024-04-28T12:37:58.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.663+0000 7f4066a00640 1 -- 172.21.15.131:0/279549402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406013c620 con 0x7f40601468c0 2024-04-28T12:37:58.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.664+0000 7f4066a00640 1 -- 172.21.15.131:0/279549402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4060069350 con 0x7f40601468c0 2024-04-28T12:37:58.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.665+0000 7f4057e00640 1 -- 172.21.15.131:0/279549402 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f404c02ad10 con 0x7f40601468c0 2024-04-28T12:37:58.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.666+0000 7f4057e00640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f404006c8f0 0x7f404006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:58.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.666+0000 7f4057e00640 1 -- 172.21.15.131:0/279549402 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f404c0ac480 con 0x7f40601468c0 2024-04-28T12:37:58.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.666+0000 7f405fe00640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f404006c8f0 0x7f404006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:58.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.667+0000 7f405fe00640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f404006c8f0 0x7f404006edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4050009a30 tx=0x7f4050008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:58.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.670+0000 7f4057e00640 1 -- 172.21.15.131:0/279549402 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f404c07a4d0 con 0x7f40601468c0 2024-04-28T12:37:58.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.874+0000 7f4066a00640 1 -- 172.21.15.131:0/279549402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f406013a5b0 con 0x7f40601468c0 2024-04-28T12:37:58.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.875+0000 7f4057e00640 1 -- 172.21.15.131:0/279549402 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f404c079dd0 con 0x7f40601468c0 2024-04-28T12:37:58.872 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:37:58.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.878+0000 7f4066a00640 1 -- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f404006c8f0 msgr2=0x7f404006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.878+0000 7f4066a00640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f404006c8f0 0x7f404006edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4050009a30 tx=0x7f4050008040 comp rx=0 tx=0).stop 2024-04-28T12:37:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.878+0000 7f4066a00640 1 -- 172.21.15.131:0/279549402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601468c0 msgr2=0x7f4060143a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.878+0000 7f4066a00640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601468c0 0x7f4060143a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f404c000c00 tx=0x7f404c0046c0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.878+0000 7f4066a00640 1 -- 172.21.15.131:0/279549402 shutdown_connections 2024-04-28T12:37:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.878+0000 7f4066a00640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f404006c8f0 0x7f404006edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.878+0000 7f4066a00640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f406015d5e0 0x7f406013a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.878+0000 7f4066a00640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40601468c0 0x7f4060143a60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.878+0000 7f4066a00640 1 --2- 172.21.15.131:0/279549402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4060145d80 0x7f40601f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.878+0000 7f4066a00640 1 -- 172.21.15.131:0/279549402 >> 172.21.15.131:0/279549402 conn(0x7f40600b2900 msgr2=0x7f4060072380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.878+0000 7f4066a00640 1 -- 172.21.15.131:0/279549402 shutdown_connections 2024-04-28T12:37:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:58.878+0000 7f4066a00640 1 -- 172.21.15.131:0/279549402 wait complete. 2024-04-28T12:37:58.895 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:37:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.091+0000 7fc4a1000640 1 Processor -- start 2024-04-28T12:37:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.091+0000 7fc4a1000640 1 -- start start 2024-04-28T12:37:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.093+0000 7fc4a1000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c15b980 0x7fc49c06ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.093+0000 7fc4a1000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc49c15bd60 0x7fc49c06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.093+0000 7fc4a1000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc49c0b4610 0x7fc49c0b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.093+0000 7fc4a1000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc49c15d060 con 0x7fc49c0b4610 2024-04-28T12:37:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.093+0000 7fc4a1000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc49c0b4fe0 con 0x7fc49c15bd60 2024-04-28T12:37:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.093+0000 7fc4a1000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc49c15c2a0 con 0x7fc49c15b980 2024-04-28T12:37:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.093+0000 7fc49be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc49c0b4610 0x7fc49c0b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.093+0000 7fc49be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc49c0b4610 0x7fc49c0b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45536/0 (socket says 172.21.15.131:45536) 2024-04-28T12:37:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.093+0000 7fc49be00640 1 -- 172.21.15.131:0/845294032 learned_addr learned my addr 172.21.15.131:0/845294032 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.094+0000 7fc49aa00640 1 --2- 172.21.15.131:0/845294032 >> v2:172.21.15.157:3300/0 conn(0x7fc49c15bd60 0x7fc49c06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.094+0000 7fc49b400640 1 --2- 172.21.15.131:0/845294032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c15b980 0x7fc49c06ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.094+0000 7fc49be00640 1 -- 172.21.15.131:0/845294032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c15b980 msgr2=0x7fc49c06ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.094+0000 7fc49be00640 1 --2- 172.21.15.131:0/845294032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c15b980 0x7fc49c06ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.094+0000 7fc49be00640 1 -- 172.21.15.131:0/845294032 >> v2:172.21.15.157:3300/0 conn(0x7fc49c15bd60 msgr2=0x7fc49c06f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:59.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.094+0000 7fc49be00640 1 --2- 172.21.15.131:0/845294032 >> v2:172.21.15.157:3300/0 conn(0x7fc49c15bd60 0x7fc49c06f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.094+0000 7fc49be00640 1 -- 172.21.15.131:0/845294032 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c15c580 con 0x7fc49c0b4610 2024-04-28T12:37:59.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.094+0000 7fc49b400640 1 --2- 172.21.15.131:0/845294032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c15b980 0x7fc49c06ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:59.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.094+0000 7fc49aa00640 1 --2- 172.21.15.131:0/845294032 >> v2:172.21.15.157:3300/0 conn(0x7fc49c15bd60 0x7fc49c06f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.095+0000 7fc49be00640 1 --2- 172.21.15.131:0/845294032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc49c0b4610 0x7fc49c0b4aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0d1870 tx=0x7fc49402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70cf776ec6425f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.095+0000 7fc49a000640 1 -- 172.21.15.131:0/845294032 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494036070 con 0x7fc49c0b4610 2024-04-28T12:37:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.095+0000 7fc49a000640 1 -- 172.21.15.131:0/845294032 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc494031600 con 0x7fc49c0b4610 2024-04-28T12:37:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.095+0000 7fc49a000640 1 -- 172.21.15.131:0/845294032 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940319d0 con 0x7fc49c0b4610 2024-04-28T12:37:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.096+0000 7fc4a1000640 1 -- 172.21.15.131:0/845294032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc49c0b4610 msgr2=0x7fc49c0b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.096+0000 7fc4a1000640 1 --2- 172.21.15.131:0/845294032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc49c0b4610 0x7fc49c0b4aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0d1870 tx=0x7fc49402a6a0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.096+0000 7fc4a1000640 1 -- 172.21.15.131:0/845294032 shutdown_connections 2024-04-28T12:37:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.096+0000 7fc4a1000640 1 --2- 172.21.15.131:0/845294032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc49c0b4610 0x7fc49c0b4aa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.096+0000 7fc4a1000640 1 --2- 172.21.15.131:0/845294032 >> v2:172.21.15.157:3300/0 conn(0x7fc49c15bd60 0x7fc49c06f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.096+0000 7fc4a1000640 1 --2- 172.21.15.131:0/845294032 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c15b980 0x7fc49c06ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.096+0000 7fc4a1000640 1 -- 172.21.15.131:0/845294032 >> 172.21.15.131:0/845294032 conn(0x7fc49c068090 msgr2=0x7fc49c0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.096+0000 7fc4a1000640 1 -- 172.21.15.131:0/845294032 shutdown_connections 2024-04-28T12:37:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.096+0000 7fc4a1000640 1 -- 172.21.15.131:0/845294032 wait complete. 2024-04-28T12:37:59.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.098+0000 7fc4a1000640 1 Processor -- start 2024-04-28T12:37:59.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.098+0000 7fc4a1000640 1 -- start start 2024-04-28T12:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.099+0000 7fc4a1000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0b4610 0x7fc49c14e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.099+0000 7fc4a1000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c15b980 0x7fc49c0cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.099+0000 7fc4a1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc49c15bd60 0x7fc49c0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.099+0000 7fc4a1000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c0b52b0 con 0x7fc49c15bd60 2024-04-28T12:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.099+0000 7fc4a1000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c15cb40 con 0x7fc49c15b980 2024-04-28T12:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.099+0000 7fc4a1000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc49c072cd0 con 0x7fc49c0b4610 2024-04-28T12:37:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.099+0000 7fc49be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc49c15bd60 0x7fc49c0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.099+0000 7fc49be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc49c15bd60 0x7fc49c0b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45542/0 (socket says 172.21.15.131:45542) 2024-04-28T12:37:59.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.099+0000 7fc49be00640 1 -- 172.21.15.131:0/1761847594 learned_addr learned my addr 172.21.15.131:0/1761847594 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:59.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.099+0000 7fc49b400640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0b4610 0x7fc49c14e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.099+0000 7fc49aa00640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c15b980 0x7fc49c0cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.100+0000 7fc49be00640 1 -- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0b4610 msgr2=0x7fc49c14e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:59.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.100+0000 7fc49be00640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0b4610 0x7fc49c14e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.100+0000 7fc49be00640 1 -- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c15b980 msgr2=0x7fc49c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:59.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.100+0000 7fc49be00640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c15b980 0x7fc49c0cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.100+0000 7fc49be00640 1 -- 172.21.15.131:0/1761847594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c14e7a0 con 0x7fc49c15bd60 2024-04-28T12:37:59.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.100+0000 7fc49aa00640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c15b980 0x7fc49c0cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:37:59.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.100+0000 7fc49be00640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc49c15bd60 0x7fc49c0b0540 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc49402ac90 tx=0x7fc494004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:59.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.100+0000 7fc493400640 1 -- 172.21.15.131:0/1761847594 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49403f020 con 0x7fc49c15bd60 2024-04-28T12:37:59.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.100+0000 7fc493400640 1 -- 172.21.15.131:0/1761847594 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc494037040 con 0x7fc49c15bd60 2024-04-28T12:37:59.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.100+0000 7fc493400640 1 -- 172.21.15.131:0/1761847594 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494036070 con 0x7fc49c15bd60 2024-04-28T12:37:59.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.100+0000 7fc4a1000640 1 -- 172.21.15.131:0/1761847594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c0d0680 con 0x7fc49c15bd60 2024-04-28T12:37:59.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.100+0000 7fc4a1000640 1 -- 172.21.15.131:0/1761847594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c0afd40 con 0x7fc49c15bd60 2024-04-28T12:37:59.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.101+0000 7fc4a1000640 1 -- 172.21.15.131:0/1761847594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49c0506c0 con 0x7fc49c15bd60 2024-04-28T12:37:59.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.102+0000 7fc493400640 1 -- 172.21.15.131:0/1761847594 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc494042020 con 0x7fc49c15bd60 2024-04-28T12:37:59.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.103+0000 7fc493400640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc47c06c8f0 0x7fc47c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.103+0000 7fc493400640 1 -- 172.21.15.131:0/1761847594 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc4940ac7f0 con 0x7fc49c15bd60 2024-04-28T12:37:59.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.104+0000 7fc49b400640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc47c06c8f0 0x7fc47c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.105+0000 7fc49b400640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc47c06c8f0 0x7fc47c06edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00a990 tx=0x7fc48c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:59.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.108+0000 7fc493400640 1 -- 172.21.15.131:0/1761847594 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc494049020 con 0x7fc49c15bd60 2024-04-28T12:37:59.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.307+0000 7fc4a1000640 1 -- 172.21.15.131:0/1761847594 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc49c0d1420 con 0x7fc49c15bd60 2024-04-28T12:37:59.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.308+0000 7fc493400640 1 -- 172.21.15.131:0/1761847594 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fc49c14e7a0 con 0x7fc49c15bd60 2024-04-28T12:37:59.307 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:37:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.310+0000 7fc4a1000640 1 -- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc47c06c8f0 msgr2=0x7fc47c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.310+0000 7fc4a1000640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc47c06c8f0 0x7fc47c06edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00a990 tx=0x7fc48c008040 comp rx=0 tx=0).stop 2024-04-28T12:37:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.310+0000 7fc4a1000640 1 -- 172.21.15.131:0/1761847594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc49c15bd60 msgr2=0x7fc49c0b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.310+0000 7fc4a1000640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc49c15bd60 0x7fc49c0b0540 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc49402ac90 tx=0x7fc494004a40 comp rx=0 tx=0).stop 2024-04-28T12:37:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.310+0000 7fc4a1000640 1 -- 172.21.15.131:0/1761847594 shutdown_connections 2024-04-28T12:37:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.310+0000 7fc4a1000640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc47c06c8f0 0x7fc47c06edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.310+0000 7fc4a1000640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc49c15bd60 0x7fc49c0b0540 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.310+0000 7fc4a1000640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49c15b980 0x7fc49c0cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.310+0000 7fc4a1000640 1 --2- 172.21.15.131:0/1761847594 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49c0b4610 0x7fc49c14e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.310+0000 7fc4a1000640 1 -- 172.21.15.131:0/1761847594 >> 172.21.15.131:0/1761847594 conn(0x7fc49c068090 msgr2=0x7fc49c15ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.310+0000 7fc4a1000640 1 -- 172.21.15.131:0/1761847594 shutdown_connections 2024-04-28T12:37:59.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.310+0000 7fc4a1000640 1 -- 172.21.15.131:0/1761847594 wait complete. 2024-04-28T12:37:59.326 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:37:59.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.531+0000 7f9d40200640 1 Processor -- start 2024-04-28T12:37:59.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.531+0000 7f9d40200640 1 -- start start 2024-04-28T12:37:59.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.533+0000 7f9d40200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d3814ceb0 0x7f9d380b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.533+0000 7f9d40200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9d380cfe60 0x7f9d3816d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.533+0000 7f9d40200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d3814d290 con 0x7f9d380d1570 2024-04-28T12:37:59.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.533+0000 7f9d40200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9d380b5140 con 0x7f9d380cfe60 2024-04-28T12:37:59.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.533+0000 7f9d40200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9d380b5420 con 0x7f9d3814ceb0 2024-04-28T12:37:59.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.533+0000 7f9d3e200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9d380cfe60 0x7f9d3816d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.533+0000 7f9d3ce00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9d380d1570 0x7f9d380cf750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33266/0 (socket says 172.21.15.131:33266) 2024-04-28T12:37:59.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.533+0000 7f9d3ce00640 1 -- 172.21.15.131:0/1038173375 learned_addr learned my addr 172.21.15.131:0/1038173375 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:37:59.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.533+0000 7f9d3d800640 1 --2- 172.21.15.131:0/1038173375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d3814ceb0 0x7f9d380b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.534+0000 7f9d3e200640 1 -- 172.21.15.131:0/1038173375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d3814ceb0 msgr2=0x7f9d380b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:37:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.534+0000 7f9d3e200640 1 --2- 172.21.15.131:0/1038173375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d3814ceb0 0x7f9d380b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.534+0000 7f9d3e200640 1 -- 172.21.15.131:0/1038173375 >> v1:172.21.15.131:6789/0 conn(0x7f9d380d1570 legacy=0x7f9d380cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.534+0000 7f9d3e200640 1 -- 172.21.15.131:0/1038173375 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d380b5700 con 0x7f9d380cfe60 2024-04-28T12:37:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.534+0000 7f9d37e00640 1 -- 172.21.15.131:0/1038173375 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141312317 0 0) 0x7f9d3814d290 con 0x7f9d380d1570 2024-04-28T12:37:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.535+0000 7f9d3e200640 1 --2- 172.21.15.131:0/1038173375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d380cfe60 0x7f9d3816d040 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009d90 tx=0x7f9d3002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f72effd48101ac server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.535+0000 7f9d37e00640 1 -- 172.21.15.131:0/1038173375 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30007740 con 0x7f9d380cfe60 2024-04-28T12:37:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.535+0000 7f9d37e00640 1 -- 172.21.15.131:0/1038173375 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d300078e0 con 0x7f9d380cfe60 2024-04-28T12:37:59.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.535+0000 7f9d37e00640 1 -- 172.21.15.131:0/1038173375 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30007cb0 con 0x7f9d380cfe60 2024-04-28T12:37:59.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.536+0000 7f9d40200640 1 -- 172.21.15.131:0/1038173375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d380cfe60 msgr2=0x7f9d3816d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:59.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.536+0000 7f9d40200640 1 --2- 172.21.15.131:0/1038173375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d380cfe60 0x7f9d3816d040 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009d90 tx=0x7f9d3002a660 comp rx=0 tx=0).stop 2024-04-28T12:37:59.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.536+0000 7f9d40200640 1 -- 172.21.15.131:0/1038173375 shutdown_connections 2024-04-28T12:37:59.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.536+0000 7f9d40200640 1 --2- 172.21.15.131:0/1038173375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d380cfe60 0x7f9d3816d040 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.536+0000 7f9d40200640 1 --2- 172.21.15.131:0/1038173375 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d3814ceb0 0x7f9d380b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.536+0000 7f9d40200640 1 -- 172.21.15.131:0/1038173375 >> 172.21.15.131:0/1038173375 conn(0x7f9d380b2900 msgr2=0x7f9d380b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:59.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.536+0000 7f9d40200640 1 -- 172.21.15.131:0/1038173375 shutdown_connections 2024-04-28T12:37:59.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.536+0000 7f9d40200640 1 -- 172.21.15.131:0/1038173375 wait complete. 2024-04-28T12:37:59.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.538+0000 7f9d40200640 1 Processor -- start 2024-04-28T12:37:59.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.538+0000 7f9d40200640 1 -- start start 2024-04-28T12:37:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.539+0000 7f9d40200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d380cfe60 0x7f9d381f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.539+0000 7f9d40200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380d1570 0x7f9d381f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.539+0000 7f9d40200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d3814ceb0 0x7f9d380c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.539+0000 7f9d40200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9d380d19d0 con 0x7f9d3814ceb0 2024-04-28T12:37:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.539+0000 7f9d40200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9d380b5cc0 con 0x7f9d380d1570 2024-04-28T12:37:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.539+0000 7f9d40200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3806f000 con 0x7f9d380cfe60 2024-04-28T12:37:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.539+0000 7f9d3e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d3814ceb0 0x7f9d380c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.539+0000 7f9d3e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d3814ceb0 0x7f9d380c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45560/0 (socket says 172.21.15.131:45560) 2024-04-28T12:37:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.539+0000 7f9d3e200640 1 -- 172.21.15.131:0/1574435040 learned_addr learned my addr 172.21.15.131:0/1574435040 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:37:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.539+0000 7f9d3d800640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d380cfe60 0x7f9d381f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.539+0000 7f9d3ce00640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380d1570 0x7f9d381f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.540+0000 7f9d3e200640 1 -- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d380cfe60 msgr2=0x7f9d381f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.540+0000 7f9d3e200640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d380cfe60 0x7f9d381f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.540+0000 7f9d3e200640 1 -- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380d1570 msgr2=0x7f9d381f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.540+0000 7f9d3e200640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380d1570 0x7f9d381f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.540+0000 7f9d3e200640 1 -- 172.21.15.131:0/1574435040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d380c6a90 con 0x7f9d3814ceb0 2024-04-28T12:37:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.540+0000 7f9d3e200640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d3814ceb0 0x7f9d380c8610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9d3002ac50 tx=0x7f9d30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.540+0000 7f9d35600640 1 -- 172.21.15.131:0/1574435040 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30040070 con 0x7f9d3814ceb0 2024-04-28T12:37:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.540+0000 7f9d35600640 1 -- 172.21.15.131:0/1574435040 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d30031bc0 con 0x7f9d3814ceb0 2024-04-28T12:37:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.540+0000 7f9d35600640 1 -- 172.21.15.131:0/1574435040 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30045460 con 0x7f9d3814ceb0 2024-04-28T12:37:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.540+0000 7f9d40200640 1 -- 172.21.15.131:0/1574435040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d380c6d70 con 0x7f9d3814ceb0 2024-04-28T12:37:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.541+0000 7f9d40200640 1 -- 172.21.15.131:0/1574435040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d380c7330 con 0x7f9d3814ceb0 2024-04-28T12:37:59.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.542+0000 7f9d40200640 1 -- 172.21.15.131:0/1574435040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d38069350 con 0x7f9d3814ceb0 2024-04-28T12:37:59.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.542+0000 7f9d35600640 1 -- 172.21.15.131:0/1574435040 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d30039040 con 0x7f9d3814ceb0 2024-04-28T12:37:59.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.544+0000 7f9d35600640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9d1c06c8f0 0x7f9d1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:37:59.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.544+0000 7f9d35600640 1 -- 172.21.15.131:0/1574435040 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9d300ad880 con 0x7f9d3814ceb0 2024-04-28T12:37:59.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.544+0000 7f9d3d800640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9d1c06c8f0 0x7f9d1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:37:59.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.545+0000 7f9d3d800640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9d1c06c8f0 0x7f9d1c06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c006fd0 tx=0x7f9d2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:37:59.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.548+0000 7f9d35600640 1 -- 172.21.15.131:0/1574435040 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9d3007ba80 con 0x7f9d3814ceb0 2024-04-28T12:37:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.786+0000 7f9d40200640 1 -- 172.21.15.131:0/1574435040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d381f9f70 con 0x7f9d3814ceb0 2024-04-28T12:37:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.788+0000 7f9d35600640 1 -- 172.21.15.131:0/1574435040 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5191 (secure 0 0 0) 0x7f9d3007b380 con 0x7f9d3814ceb0 2024-04-28T12:37:59.785 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:37:59.785 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1064054651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1064054651},{"type":"v1","addr":"172.21.15.131:6834","nonce":1064054651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/398655652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":398655652},{"type":"v1","addr":"172.21.15.131:6835","nonce":398655652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2619661531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2619661531},{"type":"v1","addr":"172.21.15.157:6837","nonce":2619661531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:37:22.236351+0000","modified":"2024-04-28T12:37:50.533095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1937780695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1937780695},{"type":"v1","addr":"172.21.15.157:6835","nonce":1937780695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34160,"qdb_cluster":[34160]},"id":1}]} 2024-04-28T12:37:59.785 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-04-28T12:37:59.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.790+0000 7f9d40200640 1 -- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9d1c06c8f0 msgr2=0x7f9d1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:59.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.791+0000 7f9d40200640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9d1c06c8f0 0x7f9d1c06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c006fd0 tx=0x7f9d2c008040 comp rx=0 tx=0).stop 2024-04-28T12:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.791+0000 7f9d40200640 1 -- 172.21.15.131:0/1574435040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d3814ceb0 msgr2=0x7f9d380c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.791+0000 7f9d40200640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d3814ceb0 0x7f9d380c8610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9d3002ac50 tx=0x7f9d30004a40 comp rx=0 tx=0).stop 2024-04-28T12:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.791+0000 7f9d40200640 1 -- 172.21.15.131:0/1574435040 shutdown_connections 2024-04-28T12:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.791+0000 7f9d40200640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9d1c06c8f0 0x7f9d1c06edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.791+0000 7f9d40200640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d3814ceb0 0x7f9d380c8610 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.791+0000 7f9d40200640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d380d1570 0x7f9d381f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.791+0000 7f9d40200640 1 --2- 172.21.15.131:0/1574435040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d380cfe60 0x7f9d381f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.791+0000 7f9d40200640 1 -- 172.21.15.131:0/1574435040 >> 172.21.15.131:0/1574435040 conn(0x7f9d380b2900 msgr2=0x7f9d38166850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.791+0000 7f9d40200640 1 -- 172.21.15.131:0/1574435040 shutdown_connections 2024-04-28T12:37:59.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:37:59.791+0000 7f9d40200640 1 -- 172.21.15.131:0/1574435040 wait complete. 2024-04-28T12:37:59.806 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.007+0000 7fcb99800640 1 Processor -- start 2024-04-28T12:38:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.007+0000 7fcb99800640 1 -- start start 2024-04-28T12:38:00.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.008+0000 7fcb99800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcb94072b70 0x7fcb940d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.008+0000 7fcb99800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb940d1060 0x7fcb9406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.008+0000 7fcb99800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb94146ba0 con 0x7fcb941467c0 2024-04-28T12:38:00.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.008+0000 7fcb99800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcb94073dd0 con 0x7fcb94072b70 2024-04-28T12:38:00.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.008+0000 7fcb99800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcb940731c0 con 0x7fcb940d1060 2024-04-28T12:38:00.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.009+0000 7fcb92a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcb941467c0 0x7fcb940b3ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33280/0 (socket says 172.21.15.131:33280) 2024-04-28T12:38:00.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.009+0000 7fcb92a00640 1 -- 172.21.15.131:0/4094726849 learned_addr learned my addr 172.21.15.131:0/4094726849 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:00.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.009+0000 7fcb93400640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb940d1060 0x7fcb9406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.009+0000 7fcb92000640 1 --2- 172.21.15.131:0/4094726849 >> v2:172.21.15.157:3300/0 conn(0x7fcb94072b70 0x7fcb940d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.009+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4008687852 0 0) 0x7fcb94146ba0 con 0x7fcb941467c0 2024-04-28T12:38:00.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.009+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb78003520 con 0x7fcb941467c0 2024-04-28T12:38:00.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.009+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4077146482 0 0) 0x7fcb78003520 con 0x7fcb941467c0 2024-04-28T12:38:00.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.009+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb940d1060 msgr2=0x7fcb9406dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:00.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.009+0000 7fcb91600640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb940d1060 0x7fcb9406dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.009+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 >> v2:172.21.15.157:3300/0 conn(0x7fcb94072b70 msgr2=0x7fcb940d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:00.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.009+0000 7fcb91600640 1 --2- 172.21.15.131:0/4094726849 >> v2:172.21.15.157:3300/0 conn(0x7fcb94072b70 0x7fcb940d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.009+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb940734a0 con 0x7fcb941467c0 2024-04-28T12:38:00.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fcb84001670 con 0x7fcb941467c0 2024-04-28T12:38:00.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 >> v1:172.21.15.131:6789/0 conn(0x7fcb941467c0 legacy=0x7fcb940b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb91600640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb940d1060 0x7fcb780039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb91600640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb78003f30 0x7fcb78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb91600640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb78004d40 0x7fcb780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb94073dd0 con 0x7fcb78004d40 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb940731c0 con 0x7fcb78003f30 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcb7800b9c0 con 0x7fcb940d1060 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb93400640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb78004d40 0x7fcb780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb92a00640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb940d1060 0x7fcb780039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb93400640 1 -- 172.21.15.131:0/4094726849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb940d1060 msgr2=0x7fcb780039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb93400640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb940d1060 0x7fcb780039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb92000640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb78003f30 0x7fcb78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb93400640 1 -- 172.21.15.131:0/4094726849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb78003f30 msgr2=0x7fcb78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb93400640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb78003f30 0x7fcb78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb93400640 1 -- 172.21.15.131:0/4094726849 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb78003520 con 0x7fcb78004d40 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.010+0000 7fcb92000640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb78003f30 0x7fcb78004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.011+0000 7fcb93400640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb78004d40 0x7fcb780051c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009fc0 tx=0x7fcb78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aec9db3fd19615c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.012+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c009c10 con 0x7fcb78004d40 2024-04-28T12:38:00.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.012+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8c036070 con 0x7fcb78004d40 2024-04-28T12:38:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.012+0000 7fcb91600640 1 -- 172.21.15.131:0/4094726849 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c031670 con 0x7fcb78004d40 2024-04-28T12:38:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.012+0000 7fcb99800640 1 -- 172.21.15.131:0/4094726849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb78004d40 msgr2=0x7fcb780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.012+0000 7fcb99800640 1 --2- 172.21.15.131:0/4094726849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb78004d40 0x7fcb780051c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009fc0 tx=0x7fcb78005830 comp rx=0 tx=0).stop 2024-04-28T12:38:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.012+0000 7fcb93400640 1 -- 172.21.15.131:0/4094726849 reap_dead start 2024-04-28T12:38:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.012+0000 7fcb99800640 1 -- 172.21.15.131:0/4094726849 shutdown_connections 2024-04-28T12:38:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.012+0000 7fcb99800640 1 -- 172.21.15.131:0/4094726849 >> 172.21.15.131:0/4094726849 conn(0x7fcb940b2900 msgr2=0x7fcb940b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.012+0000 7fcb99800640 1 -- 172.21.15.131:0/4094726849 shutdown_connections 2024-04-28T12:38:00.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.012+0000 7fcb99800640 1 -- 172.21.15.131:0/4094726849 wait complete. 2024-04-28T12:38:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.014+0000 7fcb99800640 1 Processor -- start 2024-04-28T12:38:00.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.014+0000 7fcb99800640 1 -- start start 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.015+0000 7fcb99800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb78004d40 0x7fcb940afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.015+0000 7fcb99800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb941f8040 0x7fcb941f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.015+0000 7fcb99800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb9413c640 0x7fcb9413cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.015+0000 7fcb99800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9406dff0 con 0x7fcb9413c640 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.015+0000 7fcb99800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb94073a60 con 0x7fcb78004d40 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.015+0000 7fcb99800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9406e170 con 0x7fcb941f8040 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.015+0000 7fcb93400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb9413c640 0x7fcb9413cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.015+0000 7fcb93400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb9413c640 0x7fcb9413cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45584/0 (socket says 172.21.15.131:45584) 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.015+0000 7fcb93400640 1 -- 172.21.15.131:0/3089642660 learned_addr learned my addr 172.21.15.131:0/3089642660 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.016+0000 7fcb92000640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb941f8040 0x7fcb941f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.016+0000 7fcb92a00640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb78004d40 0x7fcb940afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.016+0000 7fcb93400640 1 -- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb941f8040 msgr2=0x7fcb941f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.016+0000 7fcb93400640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb941f8040 0x7fcb941f84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.016+0000 7fcb93400640 1 -- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb78004d40 msgr2=0x7fcb940afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.016+0000 7fcb93400640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb78004d40 0x7fcb940afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.016+0000 7fcb93400640 1 -- 172.21.15.131:0/3089642660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb940aec80 con 0x7fcb9413c640 2024-04-28T12:38:00.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.016+0000 7fcb93400640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb9413c640 0x7fcb9413cad0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c02aed0 tx=0x7fcb8c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:00.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.016+0000 7fcb8aa00640 1 -- 172.21.15.131:0/3089642660 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c009c10 con 0x7fcb9413c640 2024-04-28T12:38:00.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.016+0000 7fcb8aa00640 1 -- 172.21.15.131:0/3089642660 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8c031db0 con 0x7fcb9413c640 2024-04-28T12:38:00.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.016+0000 7fcb8aa00640 1 -- 172.21.15.131:0/3089642660 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c044670 con 0x7fcb9413c640 2024-04-28T12:38:00.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.016+0000 7fcb99800640 1 -- 172.21.15.131:0/3089642660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb940aef60 con 0x7fcb9413c640 2024-04-28T12:38:00.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.017+0000 7fcb99800640 1 -- 172.21.15.131:0/3089642660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb940af520 con 0x7fcb9413c640 2024-04-28T12:38:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.018+0000 7fcb99800640 1 -- 172.21.15.131:0/3089642660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb940b0260 con 0x7fcb9413c640 2024-04-28T12:38:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.018+0000 7fcb8aa00640 1 -- 172.21.15.131:0/3089642660 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb8c005d10 con 0x7fcb9413c640 2024-04-28T12:38:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.019+0000 7fcb8aa00640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcb7406c8f0 0x7fcb7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.019+0000 7fcb8aa00640 1 -- 172.21.15.131:0/3089642660 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcb8c0acff0 con 0x7fcb9413c640 2024-04-28T12:38:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.020+0000 7fcb92a00640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcb7406c8f0 0x7fcb7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.021+0000 7fcb92a00640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcb7406c8f0 0x7fcb7406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcb84001eb0 tx=0x7fcb84001150 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:00.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.024+0000 7fcb8aa00640 1 -- 172.21.15.131:0/3089642660 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb8c07b210 con 0x7fcb9413c640 2024-04-28T12:38:00.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.225+0000 7fcb99800640 1 -- 172.21.15.131:0/3089642660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb94069350 con 0x7fcb9413c640 2024-04-28T12:38:00.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.227+0000 7fcb8aa00640 1 -- 172.21.15.131:0/3089642660 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7fcb8c039e60 con 0x7fcb9413c640 2024-04-28T12:38:00.224 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:00.224 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:00.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.229+0000 7fcb99800640 1 -- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcb7406c8f0 msgr2=0x7fcb7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.229+0000 7fcb99800640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcb7406c8f0 0x7fcb7406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcb84001eb0 tx=0x7fcb84001150 comp rx=0 tx=0).stop 2024-04-28T12:38:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.229+0000 7fcb99800640 1 -- 172.21.15.131:0/3089642660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb9413c640 msgr2=0x7fcb9413cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.229+0000 7fcb99800640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb9413c640 0x7fcb9413cad0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c02aed0 tx=0x7fcb8c009750 comp rx=0 tx=0).stop 2024-04-28T12:38:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.230+0000 7fcb99800640 1 -- 172.21.15.131:0/3089642660 shutdown_connections 2024-04-28T12:38:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.230+0000 7fcb99800640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcb7406c8f0 0x7fcb7406edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.230+0000 7fcb99800640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb9413c640 0x7fcb9413cad0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.230+0000 7fcb99800640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb941f8040 0x7fcb941f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.230+0000 7fcb99800640 1 --2- 172.21.15.131:0/3089642660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb78004d40 0x7fcb940afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.230+0000 7fcb99800640 1 -- 172.21.15.131:0/3089642660 >> 172.21.15.131:0/3089642660 conn(0x7fcb940b2900 msgr2=0x7fcb940b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.230+0000 7fcb99800640 1 -- 172.21.15.131:0/3089642660 shutdown_connections 2024-04-28T12:38:00.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.230+0000 7fcb99800640 1 -- 172.21.15.131:0/3089642660 wait complete. 2024-04-28T12:38:00.245 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:00.245 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:00.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.441+0000 7fde40a00640 1 Processor -- start 2024-04-28T12:38:00.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.441+0000 7fde40a00640 1 -- start start 2024-04-28T12:38:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.442+0000 7fde40a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fde3c0b54b0 0x7fde3c1604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.442+0000 7fde40a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde3c0b3d80 con 0x7fde3c06df40 2024-04-28T12:38:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.442+0000 7fde40a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde3c0b4060 con 0x7fde3c1458e0 2024-04-28T12:38:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.442+0000 7fde40a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fde3c0b4340 con 0x7fde3c0b54b0 2024-04-28T12:38:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.443+0000 7fde33e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fde3c0b54b0 0x7fde3c1604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.443+0000 7fde33e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fde3c0b54b0 0x7fde3c1604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45586/0 (socket says 172.21.15.131:45586) 2024-04-28T12:38:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.443+0000 7fde33e00640 1 -- 172.21.15.131:0/913599409 learned_addr learned my addr 172.21.15.131:0/913599409 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.443+0000 7fde33e00640 1 -- 172.21.15.131:0/913599409 >> v1:172.21.15.157:6789/0 conn(0x7fde3c1458e0 legacy=0x7fde3c0b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.443+0000 7fde33e00640 1 -- 172.21.15.131:0/913599409 >> v1:172.21.15.131:6789/0 conn(0x7fde3c06df40 legacy=0x7fde3c15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.443+0000 7fde33e00640 1 -- 172.21.15.131:0/913599409 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c0b4620 con 0x7fde3c0b54b0 2024-04-28T12:38:00.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.443+0000 7fde39400640 1 -- 172.21.15.131:0/913599409 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 668451737 0 0) 0x7fde3c0b3d80 con 0x7fde3c06df40 2024-04-28T12:38:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.444+0000 7fde33e00640 1 --2- 172.21.15.131:0/913599409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde3c0b54b0 0x7fde3c1604d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fde24009870 tx=0x7fde2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8ee8c40c6ef7ff server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.444+0000 7fde39400640 1 -- 172.21.15.131:0/913599409 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24037070 con 0x7fde3c0b54b0 2024-04-28T12:38:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.445+0000 7fde39400640 1 -- 172.21.15.131:0/913599409 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde24032440 con 0x7fde3c0b54b0 2024-04-28T12:38:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.445+0000 7fde39400640 1 -- 172.21.15.131:0/913599409 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24032810 con 0x7fde3c0b54b0 2024-04-28T12:38:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.445+0000 7fde40a00640 1 -- 172.21.15.131:0/913599409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde3c0b54b0 msgr2=0x7fde3c1604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.445+0000 7fde40a00640 1 --2- 172.21.15.131:0/913599409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde3c0b54b0 0x7fde3c1604d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fde24009870 tx=0x7fde2402a270 comp rx=0 tx=0).stop 2024-04-28T12:38:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.445+0000 7fde40a00640 1 -- 172.21.15.131:0/913599409 shutdown_connections 2024-04-28T12:38:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.445+0000 7fde40a00640 1 --2- 172.21.15.131:0/913599409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde3c0b54b0 0x7fde3c1604d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.445+0000 7fde40a00640 1 -- 172.21.15.131:0/913599409 >> 172.21.15.131:0/913599409 conn(0x7fde3c0b2900 msgr2=0x7fde3c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.445+0000 7fde40a00640 1 -- 172.21.15.131:0/913599409 shutdown_connections 2024-04-28T12:38:00.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.445+0000 7fde40a00640 1 -- 172.21.15.131:0/913599409 wait complete. 2024-04-28T12:38:00.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.447+0000 7fde40a00640 1 Processor -- start 2024-04-28T12:38:00.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.447+0000 7fde40a00640 1 -- start start 2024-04-28T12:38:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.448+0000 7fde40a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde3c06df40 0x7fde3c1f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.448+0000 7fde40a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde3c0b54b0 0x7fde3c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.448+0000 7fde40a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde3c1458e0 0x7fde3c13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.448+0000 7fde40a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c15df90 con 0x7fde3c06df40 2024-04-28T12:38:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.448+0000 7fde40a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c15de10 con 0x7fde3c1458e0 2024-04-28T12:38:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.448+0000 7fde40a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fde3c15e110 con 0x7fde3c0b54b0 2024-04-28T12:38:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.448+0000 7fde39e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde3c06df40 0x7fde3c1f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.448+0000 7fde39e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde3c06df40 0x7fde3c1f2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45590/0 (socket says 172.21.15.131:45590) 2024-04-28T12:38:00.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.448+0000 7fde39e00640 1 -- 172.21.15.131:0/1506574147 learned_addr learned my addr 172.21.15.131:0/1506574147 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:00.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.449+0000 7fde3a800640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde3c1458e0 0x7fde3c13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.449+0000 7fde33e00640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde3c0b54b0 0x7fde3c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.449+0000 7fde39e00640 1 -- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde3c0b54b0 msgr2=0x7fde3c15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.449+0000 7fde39e00640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde3c0b54b0 0x7fde3c15b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.449+0000 7fde39e00640 1 -- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde3c1458e0 msgr2=0x7fde3c13b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.449+0000 7fde39e00640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde3c1458e0 0x7fde3c13b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.449+0000 7fde39e00640 1 -- 172.21.15.131:0/1506574147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c1f34a0 con 0x7fde3c06df40 2024-04-28T12:38:00.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.449+0000 7fde39e00640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde3c06df40 0x7fde3c1f2f60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00c430 tx=0x7fde2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:00.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.449+0000 7fde32000640 1 -- 172.21.15.131:0/1506574147 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c017070 con 0x7fde3c06df40 2024-04-28T12:38:00.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.449+0000 7fde32000640 1 -- 172.21.15.131:0/1506574147 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde2c0052d0 con 0x7fde3c06df40 2024-04-28T12:38:00.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.449+0000 7fde32000640 1 -- 172.21.15.131:0/1506574147 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c003830 con 0x7fde3c06df40 2024-04-28T12:38:00.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.449+0000 7fde40a00640 1 -- 172.21.15.131:0/1506574147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3c0724c0 con 0x7fde3c06df40 2024-04-28T12:38:00.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.450+0000 7fde40a00640 1 -- 172.21.15.131:0/1506574147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde3c072a80 con 0x7fde3c06df40 2024-04-28T12:38:00.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.451+0000 7fde40a00640 1 -- 172.21.15.131:0/1506574147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde3c069350 con 0x7fde3c06df40 2024-04-28T12:38:00.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.451+0000 7fde32000640 1 -- 172.21.15.131:0/1506574147 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde2c01a070 con 0x7fde3c06df40 2024-04-28T12:38:00.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.452+0000 7fde32000640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fde1806c8f0 0x7fde1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.452+0000 7fde32000640 1 -- 172.21.15.131:0/1506574147 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fde2c08b120 con 0x7fde3c06df40 2024-04-28T12:38:00.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.453+0000 7fde33e00640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fde1806c8f0 0x7fde1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.454+0000 7fde33e00640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fde1806c8f0 0x7fde1806edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fde24034000 tx=0x7fde2402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:00.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.456+0000 7fde32000640 1 -- 172.21.15.131:0/1506574147 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fde2c0592a0 con 0x7fde3c06df40 2024-04-28T12:38:00.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.639+0000 7fde40a00640 1 -- 172.21.15.131:0/1506574147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fde3c06e5b0 con 0x7fde3c06df40 2024-04-28T12:38:00.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.640+0000 7fde32000640 1 -- 172.21.15.131:0/1506574147 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fde2c058ba0 con 0x7fde3c06df40 2024-04-28T12:38:00.637 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.643+0000 7fde40a00640 1 -- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fde1806c8f0 msgr2=0x7fde1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.643+0000 7fde40a00640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fde1806c8f0 0x7fde1806edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fde24034000 tx=0x7fde2402e000 comp rx=0 tx=0).stop 2024-04-28T12:38:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.643+0000 7fde40a00640 1 -- 172.21.15.131:0/1506574147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde3c06df40 msgr2=0x7fde3c1f2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.643+0000 7fde40a00640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde3c06df40 0x7fde3c1f2f60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00c430 tx=0x7fde2c00c8f0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.643+0000 7fde40a00640 1 -- 172.21.15.131:0/1506574147 shutdown_connections 2024-04-28T12:38:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.643+0000 7fde40a00640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fde1806c8f0 0x7fde1806edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.643+0000 7fde40a00640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde3c1458e0 0x7fde3c13b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.643+0000 7fde40a00640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde3c0b54b0 0x7fde3c15b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.643+0000 7fde40a00640 1 --2- 172.21.15.131:0/1506574147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde3c06df40 0x7fde3c1f2f60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.643+0000 7fde40a00640 1 -- 172.21.15.131:0/1506574147 >> 172.21.15.131:0/1506574147 conn(0x7fde3c0b2900 msgr2=0x7fde3c15e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.643+0000 7fde40a00640 1 -- 172.21.15.131:0/1506574147 shutdown_connections 2024-04-28T12:38:00.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.643+0000 7fde40a00640 1 -- 172.21.15.131:0/1506574147 wait complete. 2024-04-28T12:38:00.659 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.866+0000 7f0e6ee00640 1 Processor -- start 2024-04-28T12:38:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.866+0000 7f0e6ee00640 1 -- start start 2024-04-28T12:38:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.868+0000 7f0e6ee00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0e680d3830 0x7f0e680d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.868+0000 7f0e6ee00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e680d2d70 con 0x7f0e6815f180 2024-04-28T12:38:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.868+0000 7f0e6ee00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e680d06e0 con 0x7f0e680b3cb0 2024-04-28T12:38:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.868+0000 7f0e6ee00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0e6815f560 con 0x7f0e680d3830 2024-04-28T12:38:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.868+0000 7f0e6da00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0e6815f180 0x7f0e680d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44596/0 (socket says 172.21.15.131:44596) 2024-04-28T12:38:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.868+0000 7f0e6da00640 1 -- 172.21.15.131:0/2633530307 learned_addr learned my addr 172.21.15.131:0/2633530307 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.868+0000 7f0e6d000640 1 --2- 172.21.15.131:0/2633530307 >> v2:172.21.15.157:3300/0 conn(0x7f0e680d3830 0x7f0e680d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.868+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1932080897 0 0) 0x7f0e680d2d70 con 0x7f0e6815f180 2024-04-28T12:38:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.868+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e50003520 con 0x7f0e6815f180 2024-04-28T12:38:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2533613589 0 0) 0x7f0e50003520 con 0x7f0e6815f180 2024-04-28T12:38:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 >> v2:172.21.15.157:3300/0 conn(0x7f0e680d3830 msgr2=0x7f0e680d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 --2- 172.21.15.131:0/2633530307 >> v2:172.21.15.157:3300/0 conn(0x7f0e680d3830 0x7f0e680d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 >> v1:172.21.15.157:6789/0 conn(0x7f0e680b3cb0 legacy=0x7f0e680b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e680d09c0 con 0x7f0e6815f180 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f0e5c001670 con 0x7f0e6815f180 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 >> v1:172.21.15.131:6789/0 conn(0x7f0e6815f180 legacy=0x7f0e680d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 --2- 172.21.15.131:0/2633530307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e50003a20 0x7f0e50003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 --2- 172.21.15.131:0/2633530307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e50004800 0x7f0e50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 --2- 172.21.15.131:0/2633530307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 0x7f0e5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6815f560 con 0x7f0e500051c0 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5000c0f0 con 0x7f0e50004800 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e5000c2f0 con 0x7f0e50003a20 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.869+0000 7f0e6e400640 1 --2- 172.21.15.131:0/2633530307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 0x7f0e5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.870+0000 7f0e6da00640 1 --2- 172.21.15.131:0/2633530307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e50003a20 0x7f0e50003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.870+0000 7f0e6d000640 1 --2- 172.21.15.131:0/2633530307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e50004800 0x7f0e50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.870+0000 7f0e6e400640 1 -- 172.21.15.131:0/2633530307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e50003a20 msgr2=0x7f0e50003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:00.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.870+0000 7f0e6e400640 1 --2- 172.21.15.131:0/2633530307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e50003a20 0x7f0e50003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.870+0000 7f0e6e400640 1 -- 172.21.15.131:0/2633530307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e50004800 msgr2=0x7f0e50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:00.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.870+0000 7f0e6e400640 1 --2- 172.21.15.131:0/2633530307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e50004800 0x7f0e50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.870+0000 7f0e6e400640 1 -- 172.21.15.131:0/2633530307 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e50003520 con 0x7f0e500051c0 2024-04-28T12:38:00.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.870+0000 7f0e6d000640 1 --2- 172.21.15.131:0/2633530307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e50004800 0x7f0e50004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:00.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.871+0000 7f0e6e400640 1 -- 172.21.15.131:0/2633530307 reap_dead start 2024-04-28T12:38:00.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.871+0000 7f0e6e400640 1 --2- 172.21.15.131:0/2633530307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 0x7f0e5000b9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0e60001e20 tx=0x7f0e50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dbaa50c3a87094 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:00.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.871+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60001a70 con 0x7f0e500051c0 2024-04-28T12:38:00.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.871+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6002ae40 con 0x7f0e500051c0 2024-04-28T12:38:00.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.871+0000 7f0e67e00640 1 -- 172.21.15.131:0/2633530307 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60003bb0 con 0x7f0e500051c0 2024-04-28T12:38:00.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.872+0000 7f0e6ee00640 1 -- 172.21.15.131:0/2633530307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 msgr2=0x7f0e5000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.872+0000 7f0e6ee00640 1 --2- 172.21.15.131:0/2633530307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 0x7f0e5000b9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0e60001e20 tx=0x7f0e50005830 comp rx=0 tx=0).stop 2024-04-28T12:38:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.872+0000 7f0e6ee00640 1 -- 172.21.15.131:0/2633530307 shutdown_connections 2024-04-28T12:38:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.872+0000 7f0e6ee00640 1 --2- 172.21.15.131:0/2633530307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 0x7f0e5000b9c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.872+0000 7f0e6ee00640 1 -- 172.21.15.131:0/2633530307 >> 172.21.15.131:0/2633530307 conn(0x7f0e680d4bd0 msgr2=0x7f0e680d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.872+0000 7f0e6ee00640 1 -- 172.21.15.131:0/2633530307 shutdown_connections 2024-04-28T12:38:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.872+0000 7f0e6ee00640 1 -- 172.21.15.131:0/2633530307 wait complete. 2024-04-28T12:38:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.873+0000 7f0e6ee00640 1 Processor -- start 2024-04-28T12:38:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.874+0000 7f0e6ee00640 1 -- start start 2024-04-28T12:38:00.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.874+0000 7f0e6ee00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 0x7f0e68150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.874+0000 7f0e6ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e680b4cd0 0x7f0e680b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.874+0000 7f0e6ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e680a0350 0x7f0e680a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.875+0000 7f0e6ee00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0e680d11c0 con 0x7f0e500051c0 2024-04-28T12:38:00.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.875+0000 7f0e6ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0e680d2a40 con 0x7f0e680b4cd0 2024-04-28T12:38:00.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.875+0000 7f0e6ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e680b5790 con 0x7f0e680a0350 2024-04-28T12:38:00.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.875+0000 7f0e6da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 0x7f0e68150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.875+0000 7f0e6da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 0x7f0e68150090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42840/0 (socket says 172.21.15.131:42840) 2024-04-28T12:38:00.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.875+0000 7f0e6da00640 1 -- 172.21.15.131:0/3859113890 learned_addr learned my addr 172.21.15.131:0/3859113890 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:00.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.875+0000 7f0e6da00640 1 -- 172.21.15.131:0/3859113890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e680a0350 msgr2=0x7f0e680a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.875+0000 7f0e6da00640 1 --2- 172.21.15.131:0/3859113890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e680a0350 0x7f0e680a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.875+0000 7f0e6da00640 1 -- 172.21.15.131:0/3859113890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e680b4cd0 msgr2=0x7f0e680b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:00.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.875+0000 7f0e6da00640 1 --2- 172.21.15.131:0/3859113890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e680b4cd0 0x7f0e680b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:00.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.875+0000 7f0e6da00640 1 -- 172.21.15.131:0/3859113890 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e680b5a70 con 0x7f0e500051c0 2024-04-28T12:38:00.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.875+0000 7f0e6da00640 1 --2- 172.21.15.131:0/3859113890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 0x7f0e68150090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0017e0 tx=0x7f0e5c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:00.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.876+0000 7f0e65600640 1 -- 172.21.15.131:0/3859113890 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c00ee60 con 0x7f0e500051c0 2024-04-28T12:38:00.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.876+0000 7f0e65600640 1 -- 172.21.15.131:0/3859113890 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e5c018070 con 0x7f0e500051c0 2024-04-28T12:38:00.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.876+0000 7f0e65600640 1 -- 172.21.15.131:0/3859113890 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c00cab0 con 0x7f0e500051c0 2024-04-28T12:38:00.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.876+0000 7f0e6ee00640 1 -- 172.21.15.131:0/3859113890 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e680b5d50 con 0x7f0e500051c0 2024-04-28T12:38:00.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.876+0000 7f0e6ee00640 1 -- 172.21.15.131:0/3859113890 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e680b6e20 con 0x7f0e500051c0 2024-04-28T12:38:00.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.877+0000 7f0e6ee00640 1 -- 172.21.15.131:0/3859113890 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e680506c0 con 0x7f0e500051c0 2024-04-28T12:38:00.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.878+0000 7f0e65600640 1 -- 172.21.15.131:0/3859113890 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e5c00cc50 con 0x7f0e500051c0 2024-04-28T12:38:00.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.879+0000 7f0e65600640 1 --2- 172.21.15.131:0/3859113890 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0e4806c8f0 0x7f0e4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:00.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.879+0000 7f0e65600640 1 -- 172.21.15.131:0/3859113890 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0e5c08be10 con 0x7f0e500051c0 2024-04-28T12:38:00.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.879+0000 7f0e6d000640 1 --2- 172.21.15.131:0/3859113890 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0e4806c8f0 0x7f0e4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:00.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.881+0000 7f0e6d000640 1 --2- 172.21.15.131:0/3859113890 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0e4806c8f0 0x7f0e4806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0e680ba4e0 tx=0x7f0e54008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:00.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:00.883+0000 7f0e65600640 1 -- 172.21.15.131:0/3859113890 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e5c08f030 con 0x7f0e500051c0 2024-04-28T12:38:01.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.079+0000 7f0e6ee00640 1 -- 172.21.15.131:0/3859113890 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0e6815bee0 con 0x7f0e500051c0 2024-04-28T12:38:01.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.079+0000 7f0e65600640 1 -- 172.21.15.131:0/3859113890 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f0e5c059dd0 con 0x7f0e500051c0 2024-04-28T12:38:01.076 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.082+0000 7f0e6ee00640 1 -- 172.21.15.131:0/3859113890 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0e4806c8f0 msgr2=0x7f0e4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.082+0000 7f0e6ee00640 1 --2- 172.21.15.131:0/3859113890 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0e4806c8f0 0x7f0e4806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0e680ba4e0 tx=0x7f0e54008040 comp rx=0 tx=0).stop 2024-04-28T12:38:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.082+0000 7f0e6ee00640 1 -- 172.21.15.131:0/3859113890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 msgr2=0x7f0e68150090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.082+0000 7f0e6ee00640 1 --2- 172.21.15.131:0/3859113890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 0x7f0e68150090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0017e0 tx=0x7f0e5c00ec10 comp rx=0 tx=0).stop 2024-04-28T12:38:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.082+0000 7f0e6ee00640 1 -- 172.21.15.131:0/3859113890 shutdown_connections 2024-04-28T12:38:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.082+0000 7f0e6ee00640 1 --2- 172.21.15.131:0/3859113890 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0e4806c8f0 0x7f0e4806edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.082+0000 7f0e6ee00640 1 --2- 172.21.15.131:0/3859113890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e680a0350 0x7f0e680a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.082+0000 7f0e6ee00640 1 --2- 172.21.15.131:0/3859113890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e680b4cd0 0x7f0e680b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.082+0000 7f0e6ee00640 1 --2- 172.21.15.131:0/3859113890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e500051c0 0x7f0e68150090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.082+0000 7f0e6ee00640 1 -- 172.21.15.131:0/3859113890 >> 172.21.15.131:0/3859113890 conn(0x7f0e680d4bd0 msgr2=0x7f0e680b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.082+0000 7f0e6ee00640 1 -- 172.21.15.131:0/3859113890 shutdown_connections 2024-04-28T12:38:01.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.083+0000 7f0e6ee00640 1 -- 172.21.15.131:0/3859113890 wait complete. 2024-04-28T12:38:01.098 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:01.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.306+0000 7f0218c00640 1 Processor -- start 2024-04-28T12:38:01.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.306+0000 7f0218c00640 1 -- start start 2024-04-28T12:38:01.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.308+0000 7f0218c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0214072a90 0x7f02140cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.308+0000 7f0218c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f021415d5e0 0x7f02140cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.308+0000 7f0218c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0214146160 con 0x7f0214145d80 2024-04-28T12:38:01.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.308+0000 7f0218c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f021406dfb0 con 0x7f021415d5e0 2024-04-28T12:38:01.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.308+0000 7f0218c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0214074180 con 0x7f0214072a90 2024-04-28T12:38:01.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.308+0000 7f0212800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f021415d5e0 0x7f02140cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:01.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.309+0000 7f0212800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f021415d5e0 0x7f02140cf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42844/0 (socket says 172.21.15.131:42844) 2024-04-28T12:38:01.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.309+0000 7f0212800640 1 -- 172.21.15.131:0/4045816102 learned_addr learned my addr 172.21.15.131:0/4045816102 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:01.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.309+0000 7f0212800640 1 -- 172.21.15.131:0/4045816102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0214072a90 msgr2=0x7f02140cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:01.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.309+0000 7f0212800640 1 --2- 172.21.15.131:0/4045816102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0214072a90 0x7f02140cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.309+0000 7f0212800640 1 -- 172.21.15.131:0/4045816102 >> v1:172.21.15.157:6789/0 conn(0x7f0214145d80 legacy=0x7f0214072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.309+0000 7f0212800640 1 -- 172.21.15.131:0/4045816102 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02140d0710 con 0x7f021415d5e0 2024-04-28T12:38:01.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.310+0000 7f0212800640 1 --2- 172.21.15.131:0/4045816102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021415d5e0 0x7f02140cf9a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f020c009e50 tx=0x7f020c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a2412a40420720 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:01.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.310+0000 7f0210a00640 1 -- 172.21.15.131:0/4045816102 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c036070 con 0x7f021415d5e0 2024-04-28T12:38:01.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.311+0000 7f0210a00640 1 -- 172.21.15.131:0/4045816102 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f020c031600 con 0x7f021415d5e0 2024-04-28T12:38:01.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.311+0000 7f0210a00640 1 -- 172.21.15.131:0/4045816102 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c0319d0 con 0x7f021415d5e0 2024-04-28T12:38:01.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.311+0000 7f0218c00640 1 -- 172.21.15.131:0/4045816102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021415d5e0 msgr2=0x7f02140cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.311+0000 7f0218c00640 1 --2- 172.21.15.131:0/4045816102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021415d5e0 0x7f02140cf9a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f020c009e50 tx=0x7f020c02a710 comp rx=0 tx=0).stop 2024-04-28T12:38:01.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.312+0000 7f0218c00640 1 -- 172.21.15.131:0/4045816102 shutdown_connections 2024-04-28T12:38:01.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.312+0000 7f0218c00640 1 --2- 172.21.15.131:0/4045816102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021415d5e0 0x7f02140cf9a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.312+0000 7f0218c00640 1 --2- 172.21.15.131:0/4045816102 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0214072a90 0x7f02140cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.312+0000 7f0218c00640 1 -- 172.21.15.131:0/4045816102 >> 172.21.15.131:0/4045816102 conn(0x7f02140b2900 msgr2=0x7f02140b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:01.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.312+0000 7f0218c00640 1 -- 172.21.15.131:0/4045816102 shutdown_connections 2024-04-28T12:38:01.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.312+0000 7f0218c00640 1 -- 172.21.15.131:0/4045816102 wait complete. 2024-04-28T12:38:01.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.313+0000 7f0218c00640 1 Processor -- start 2024-04-28T12:38:01.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.314+0000 7f0218c00640 1 -- start start 2024-04-28T12:38:01.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.314+0000 7f0218c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0214072a90 0x7f02140af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.314+0000 7f0218c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0214145d80 0x7f021413c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.314+0000 7f0218c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021415d5e0 0x7f021415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.314+0000 7f0218c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f02140d09f0 con 0x7f021415d5e0 2024-04-28T12:38:01.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.314+0000 7f0218c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f021406e530 con 0x7f0214145d80 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.314+0000 7f0218c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f02141468c0 con 0x7f0214072a90 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.315+0000 7f0212800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021415d5e0 0x7f021415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.315+0000 7f0212800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021415d5e0 0x7f021415aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42854/0 (socket says 172.21.15.131:42854) 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.315+0000 7f0212800640 1 -- 172.21.15.131:0/1308122699 learned_addr learned my addr 172.21.15.131:0/1308122699 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.315+0000 7f0212800640 1 -- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0214072a90 msgr2=0x7f02140af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.315+0000 7f0212800640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0214072a90 0x7f02140af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.315+0000 7f0212800640 1 -- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0214145d80 msgr2=0x7f021413c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.315+0000 7f0211400640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0214145d80 0x7f021413c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.315+0000 7f0212800640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0214145d80 0x7f021413c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.315+0000 7f0212800640 1 -- 172.21.15.131:0/1308122699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02140b1940 con 0x7f021415d5e0 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.315+0000 7f0212800640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021415d5e0 0x7f021415aff0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f020c009e50 tx=0x7f020c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.316+0000 7f020a000640 1 -- 172.21.15.131:0/1308122699 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c036050 con 0x7f021415d5e0 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.316+0000 7f020a000640 1 -- 172.21.15.131:0/1308122699 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f020c042070 con 0x7f021415d5e0 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.316+0000 7f020a000640 1 -- 172.21.15.131:0/1308122699 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c009510 con 0x7f021415d5e0 2024-04-28T12:38:01.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.316+0000 7f0218c00640 1 -- 172.21.15.131:0/1308122699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02140af7f0 con 0x7f021415d5e0 2024-04-28T12:38:01.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.316+0000 7f0218c00640 1 -- 172.21.15.131:0/1308122699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02140b1ec0 con 0x7f021415d5e0 2024-04-28T12:38:01.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.317+0000 7f0218c00640 1 -- 172.21.15.131:0/1308122699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0214069350 con 0x7f021415d5e0 2024-04-28T12:38:01.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.318+0000 7f020a000640 1 -- 172.21.15.131:0/1308122699 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f020c007bf0 con 0x7f021415d5e0 2024-04-28T12:38:01.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.319+0000 7f020a000640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f01f406c8f0 0x7f01f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.319+0000 7f020a000640 1 -- 172.21.15.131:0/1308122699 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f020c0acd90 con 0x7f021415d5e0 2024-04-28T12:38:01.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.320+0000 7f0211e00640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f01f406c8f0 0x7f01f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:01.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.321+0000 7f0211e00640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f01f406c8f0 0x7f01f406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0204000f60 tx=0x7f02040079d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:01.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.323+0000 7f020a000640 1 -- 172.21.15.131:0/1308122699 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f020c07af10 con 0x7f021415d5e0 2024-04-28T12:38:01.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.524+0000 7f0218c00640 1 -- 172.21.15.131:0/1308122699 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f02140b42a0 con 0x7f021415d5e0 2024-04-28T12:38:01.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.525+0000 7f020a000640 1 -- 172.21.15.131:0/1308122699 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f020c07a810 con 0x7f021415d5e0 2024-04-28T12:38:01.521 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.527+0000 7f0218c00640 1 -- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f01f406c8f0 msgr2=0x7f01f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.527+0000 7f0218c00640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f01f406c8f0 0x7f01f406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0204000f60 tx=0x7f02040079d0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.527+0000 7f0218c00640 1 -- 172.21.15.131:0/1308122699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021415d5e0 msgr2=0x7f021415aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.527+0000 7f0218c00640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021415d5e0 0x7f021415aff0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f020c009e50 tx=0x7f020c004a40 comp rx=0 tx=0).stop 2024-04-28T12:38:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.527+0000 7f0218c00640 1 -- 172.21.15.131:0/1308122699 shutdown_connections 2024-04-28T12:38:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.527+0000 7f0218c00640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f01f406c8f0 0x7f01f406edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.527+0000 7f0218c00640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f021415d5e0 0x7f021415aff0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.527+0000 7f0218c00640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0214145d80 0x7f021413c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.527+0000 7f0218c00640 1 --2- 172.21.15.131:0/1308122699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0214072a90 0x7f02140af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.527+0000 7f0218c00640 1 -- 172.21.15.131:0/1308122699 >> 172.21.15.131:0/1308122699 conn(0x7f02140b2900 msgr2=0x7f0214144740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:01.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.527+0000 7f0218c00640 1 -- 172.21.15.131:0/1308122699 shutdown_connections 2024-04-28T12:38:01.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.527+0000 7f0218c00640 1 -- 172.21.15.131:0/1308122699 wait complete. 2024-04-28T12:38:01.544 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:01.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.747+0000 7f0196800640 1 Processor -- start 2024-04-28T12:38:01.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.748+0000 7f0196800640 1 -- start start 2024-04-28T12:38:01.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.749+0000 7f0196800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f019015cca0 0x7f01900cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.749+0000 7f0196800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01900d12e0 con 0x7f0190160240 2024-04-28T12:38:01.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.749+0000 7f0196800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01900d0240 con 0x7f01900d1f40 2024-04-28T12:38:01.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.749+0000 7f0196800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f01900d0520 con 0x7f019015cca0 2024-04-28T12:38:01.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.750+0000 7f018ea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0190160240 0x7f01900d1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44604/0 (socket says 172.21.15.131:44604) 2024-04-28T12:38:01.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.750+0000 7f018ea00640 1 -- 172.21.15.131:0/3628547421 learned_addr learned my addr 172.21.15.131:0/3628547421 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:01.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.750+0000 7f018f400640 1 --2- 172.21.15.131:0/3628547421 >> v2:172.21.15.157:3300/0 conn(0x7f019015cca0 0x7f01900cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:01.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.750+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4146825356 0 0) 0x7f0178001110 con 0x7f0190160240 2024-04-28T12:38:01.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.750+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0174003520 con 0x7f0190160240 2024-04-28T12:38:01.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2637804984 0 0) 0x7f0174003520 con 0x7f0190160240 2024-04-28T12:38:01.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 >> v2:172.21.15.157:3300/0 conn(0x7f019015cca0 msgr2=0x7f01900cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:01.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 --2- 172.21.15.131:0/3628547421 >> v2:172.21.15.157:3300/0 conn(0x7f019015cca0 0x7f01900cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 >> v1:172.21.15.157:6789/0 conn(0x7f01900d1f40 legacy=0x7f01900d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01900d0800 con 0x7f0190160240 2024-04-28T12:38:01.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f0178001840 con 0x7f0190160240 2024-04-28T12:38:01.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 >> v1:172.21.15.131:6789/0 conn(0x7f0190160240 legacy=0x7f01900d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 --2- 172.21.15.131:0/3628547421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01740039b0 0x7f0174003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 --2- 172.21.15.131:0/3628547421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0174004800 0x7f0174004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 --2- 172.21.15.131:0/3628547421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740051c0 0x7f017400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f01900d0520 con 0x7f01740051c0 2024-04-28T12:38:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f017400c060 con 0x7f0174004800 2024-04-28T12:38:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f017400c260 con 0x7f01740039b0 2024-04-28T12:38:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018fe00640 1 --2- 172.21.15.131:0/3628547421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740051c0 0x7f017400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018ea00640 1 --2- 172.21.15.131:0/3628547421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0174004800 0x7f0174004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.751+0000 7f018f400640 1 --2- 172.21.15.131:0/3628547421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01740039b0 0x7f0174003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.752+0000 7f018fe00640 1 -- 172.21.15.131:0/3628547421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01740039b0 msgr2=0x7f0174003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.752+0000 7f018fe00640 1 --2- 172.21.15.131:0/3628547421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01740039b0 0x7f0174003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.752+0000 7f018fe00640 1 -- 172.21.15.131:0/3628547421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0174004800 msgr2=0x7f0174004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.752+0000 7f018fe00640 1 --2- 172.21.15.131:0/3628547421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0174004800 0x7f0174004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.752+0000 7f018fe00640 1 -- 172.21.15.131:0/3628547421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0174003520 con 0x7f01740051c0 2024-04-28T12:38:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.753+0000 7f018fe00640 1 -- 172.21.15.131:0/3628547421 reap_dead start 2024-04-28T12:38:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.753+0000 7f018fe00640 1 --2- 172.21.15.131:0/3628547421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740051c0 0x7f017400b9c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0188001750 tx=0x7f0174005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6718c5e57a6f56d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.753+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188001370 con 0x7f01740051c0 2024-04-28T12:38:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.753+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0188005310 con 0x7f01740051c0 2024-04-28T12:38:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.753+0000 7f018e000640 1 -- 172.21.15.131:0/3628547421 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018802abe0 con 0x7f01740051c0 2024-04-28T12:38:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.753+0000 7f0196800640 1 -- 172.21.15.131:0/3628547421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740051c0 msgr2=0x7f017400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.753+0000 7f0196800640 1 --2- 172.21.15.131:0/3628547421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740051c0 0x7f017400b9c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0188001750 tx=0x7f0174005830 comp rx=0 tx=0).stop 2024-04-28T12:38:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.754+0000 7f0196800640 1 -- 172.21.15.131:0/3628547421 shutdown_connections 2024-04-28T12:38:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.754+0000 7f0196800640 1 --2- 172.21.15.131:0/3628547421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740051c0 0x7f017400b9c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.754+0000 7f0196800640 1 -- 172.21.15.131:0/3628547421 >> 172.21.15.131:0/3628547421 conn(0x7f01900d4e40 msgr2=0x7f01900d5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.754+0000 7f0196800640 1 -- 172.21.15.131:0/3628547421 shutdown_connections 2024-04-28T12:38:01.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.754+0000 7f0196800640 1 -- 172.21.15.131:0/3628547421 wait complete. 2024-04-28T12:38:01.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.755+0000 7f0196800640 1 Processor -- start 2024-04-28T12:38:01.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.756+0000 7f0196800640 1 -- start start 2024-04-28T12:38:01.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.756+0000 7f0196800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01740051c0 0x7f01900b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.756+0000 7f0196800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01900b52d0 0x7f01900cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.756+0000 7f0196800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01900c8bc0 0x7f019014eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f0196800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f01901606b0 con 0x7f01900c8bc0 2024-04-28T12:38:01.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f0196800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01900d0dc0 con 0x7f01740051c0 2024-04-28T12:38:01.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f0196800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f019015d080 con 0x7f01900b52d0 2024-04-28T12:38:01.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f018fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01900c8bc0 0x7f019014eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:01.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f018fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01900c8bc0 0x7f019014eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42874/0 (socket says 172.21.15.131:42874) 2024-04-28T12:38:01.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f018fe00640 1 -- 172.21.15.131:0/262509666 learned_addr learned my addr 172.21.15.131:0/262509666 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:01.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f018ea00640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01900b52d0 0x7f01900cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:01.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f018f400640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01740051c0 0x7f01900b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:01.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f018fe00640 1 -- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01900b52d0 msgr2=0x7f01900cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f018fe00640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01900b52d0 0x7f01900cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f018fe00640 1 -- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01740051c0 msgr2=0x7f01900b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f018fe00640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01740051c0 0x7f01900b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f018fe00640 1 -- 172.21.15.131:0/262509666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01900b5800 con 0x7f01900c8bc0 2024-04-28T12:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.757+0000 7f018f400640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01740051c0 0x7f01900b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.758+0000 7f018fe00640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01900c8bc0 0x7f019014eea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0188001870 tx=0x7f0188001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.758+0000 7f0187400640 1 -- 172.21.15.131:0/262509666 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188001370 con 0x7f01900c8bc0 2024-04-28T12:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.758+0000 7f0187400640 1 -- 172.21.15.131:0/262509666 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0188000ed0 con 0x7f01900c8bc0 2024-04-28T12:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.758+0000 7f0187400640 1 -- 172.21.15.131:0/262509666 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0188007a30 con 0x7f01900c8bc0 2024-04-28T12:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.758+0000 7f0196800640 1 -- 172.21.15.131:0/262509666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01900b5ae0 con 0x7f01900c8bc0 2024-04-28T12:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.758+0000 7f0196800640 1 -- 172.21.15.131:0/262509666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01900b60a0 con 0x7f01900c8bc0 2024-04-28T12:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.759+0000 7f0196800640 1 -- 172.21.15.131:0/262509666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0190069350 con 0x7f01900c8bc0 2024-04-28T12:38:01.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.760+0000 7f0187400640 1 -- 172.21.15.131:0/262509666 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0188007030 con 0x7f01900c8bc0 2024-04-28T12:38:01.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.761+0000 7f0187400640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f016c06c920 0x7f016c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:01.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.761+0000 7f0187400640 1 -- 172.21.15.131:0/262509666 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f01880ab0d0 con 0x7f01900c8bc0 2024-04-28T12:38:01.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.762+0000 7f018f400640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f016c06c920 0x7f016c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:01.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.763+0000 7f018f400640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f016c06c920 0x7f016c06ede0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0180007440 tx=0x7f018000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:01.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.765+0000 7f0187400640 1 -- 172.21.15.131:0/262509666 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0188079250 con 0x7f01900c8bc0 2024-04-28T12:38:01.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.965+0000 7f0196800640 1 -- 172.21.15.131:0/262509666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f01900c84d0 con 0x7f01900c8bc0 2024-04-28T12:38:01.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.965+0000 7f0187400640 1 -- 172.21.15.131:0/262509666 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f018802ae20 con 0x7f01900c8bc0 2024-04-28T12:38:01.962 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:01.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.968+0000 7f0196800640 1 -- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f016c06c920 msgr2=0x7f016c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.968+0000 7f0196800640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f016c06c920 0x7f016c06ede0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0180007440 tx=0x7f018000b040 comp rx=0 tx=0).stop 2024-04-28T12:38:01.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.968+0000 7f0196800640 1 -- 172.21.15.131:0/262509666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01900c8bc0 msgr2=0x7f019014eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:01.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.968+0000 7f0196800640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01900c8bc0 0x7f019014eea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0188001870 tx=0x7f0188001120 comp rx=0 tx=0).stop 2024-04-28T12:38:01.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.968+0000 7f0196800640 1 -- 172.21.15.131:0/262509666 shutdown_connections 2024-04-28T12:38:01.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.968+0000 7f0196800640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f016c06c920 0x7f016c06ede0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.968+0000 7f0196800640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01900c8bc0 0x7f019014eea0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.968+0000 7f0196800640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01900b52d0 0x7f01900cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.968+0000 7f0196800640 1 --2- 172.21.15.131:0/262509666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01740051c0 0x7f01900b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:01.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.968+0000 7f0196800640 1 -- 172.21.15.131:0/262509666 >> 172.21.15.131:0/262509666 conn(0x7f01900d4e40 msgr2=0x7f019015ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:01.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.968+0000 7f0196800640 1 -- 172.21.15.131:0/262509666 shutdown_connections 2024-04-28T12:38:01.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:01.968+0000 7f0196800640 1 -- 172.21.15.131:0/262509666 wait complete. 2024-04-28T12:38:01.984 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:38:02.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.182+0000 7f4f89400640 1 Processor -- start 2024-04-28T12:38:02.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.182+0000 7f4f89400640 1 -- start start 2024-04-28T12:38:02.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.184+0000 7f4f89400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4f84072b70 0x7f4f840d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:02.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.184+0000 7f4f89400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4f840d1060 0x7f4f8406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:02.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.184+0000 7f4f89400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f84146ba0 con 0x7f4f841467c0 2024-04-28T12:38:02.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.184+0000 7f4f89400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4f84073dd0 con 0x7f4f840d1060 2024-04-28T12:38:02.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.184+0000 7f4f89400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4f840731c0 con 0x7f4f84072b70 2024-04-28T12:38:02.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.184+0000 7f4f83200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4f840d1060 0x7f4f8406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:02.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.184+0000 7f4f83200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4f840d1060 0x7f4f8406dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42876/0 (socket says 172.21.15.131:42876) 2024-04-28T12:38:02.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.184+0000 7f4f83200640 1 -- 172.21.15.131:0/4082282392 learned_addr learned my addr 172.21.15.131:0/4082282392 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:02.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.185+0000 7f4f79e00640 1 --2- 172.21.15.131:0/4082282392 >> v2:172.21.15.157:3300/0 conn(0x7f4f84072b70 0x7f4f840d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:02.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.185+0000 7f4f83200640 1 -- 172.21.15.131:0/4082282392 >> v2:172.21.15.157:3300/0 conn(0x7f4f84072b70 msgr2=0x7f4f840d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:02.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.185+0000 7f4f83200640 1 --2- 172.21.15.131:0/4082282392 >> v2:172.21.15.157:3300/0 conn(0x7f4f84072b70 0x7f4f840d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.185+0000 7f4f83200640 1 -- 172.21.15.131:0/4082282392 >> v1:172.21.15.157:6789/0 conn(0x7f4f841467c0 legacy=0x7f4f840b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:02.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.185+0000 7f4f83200640 1 -- 172.21.15.131:0/4082282392 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f840734a0 con 0x7f4f840d1060 2024-04-28T12:38:02.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.186+0000 7f4f83200640 1 --2- 172.21.15.131:0/4082282392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f840d1060 0x7f4f8406dff0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f84072320 tx=0x7f4f7c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb4d0a4870c440a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:02.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.186+0000 7f4f81e00640 1 -- 172.21.15.131:0/4082282392 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c037070 con 0x7f4f840d1060 2024-04-28T12:38:02.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.186+0000 7f4f81e00640 1 -- 172.21.15.131:0/4082282392 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f7c032440 con 0x7f4f840d1060 2024-04-28T12:38:02.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.186+0000 7f4f81e00640 1 -- 172.21.15.131:0/4082282392 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c032810 con 0x7f4f840d1060 2024-04-28T12:38:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.187+0000 7f4f89400640 1 -- 172.21.15.131:0/4082282392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f840d1060 msgr2=0x7f4f8406dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.187+0000 7f4f89400640 1 --2- 172.21.15.131:0/4082282392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f840d1060 0x7f4f8406dff0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f84072320 tx=0x7f4f7c02a230 comp rx=0 tx=0).stop 2024-04-28T12:38:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.187+0000 7f4f89400640 1 -- 172.21.15.131:0/4082282392 shutdown_connections 2024-04-28T12:38:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.187+0000 7f4f89400640 1 --2- 172.21.15.131:0/4082282392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f840d1060 0x7f4f8406dff0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.187+0000 7f4f89400640 1 --2- 172.21.15.131:0/4082282392 >> v2:172.21.15.157:3300/0 conn(0x7f4f84072b70 0x7f4f840d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.187+0000 7f4f89400640 1 -- 172.21.15.131:0/4082282392 >> 172.21.15.131:0/4082282392 conn(0x7f4f840b2900 msgr2=0x7f4f840b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.187+0000 7f4f89400640 1 -- 172.21.15.131:0/4082282392 shutdown_connections 2024-04-28T12:38:02.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.187+0000 7f4f89400640 1 -- 172.21.15.131:0/4082282392 wait complete. 2024-04-28T12:38:02.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.189+0000 7f4f89400640 1 Processor -- start 2024-04-28T12:38:02.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.189+0000 7f4f89400640 1 -- start start 2024-04-28T12:38:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.190+0000 7f4f89400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f84072b70 0x7f4f8414b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.190+0000 7f4f89400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f840d1060 0x7f4f8413bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.190+0000 7f4f89400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f841467c0 0x7f4f8413c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.190+0000 7f4f89400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8406e530 con 0x7f4f84072b70 2024-04-28T12:38:02.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.190+0000 7f4f89400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84073a60 con 0x7f4f840d1060 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.190+0000 7f4f89400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4f8406e6b0 con 0x7f4f841467c0 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.191+0000 7f4f82800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f84072b70 0x7f4f8414b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.191+0000 7f4f82800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f84072b70 0x7f4f8414b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42878/0 (socket says 172.21.15.131:42878) 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.191+0000 7f4f82800640 1 -- 172.21.15.131:0/2557895919 learned_addr learned my addr 172.21.15.131:0/2557895919 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.191+0000 7f4f82800640 1 -- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f841467c0 msgr2=0x7f4f8413c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.191+0000 7f4f82800640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f841467c0 0x7f4f8413c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.191+0000 7f4f79e00640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f840d1060 0x7f4f8413bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.191+0000 7f4f82800640 1 -- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f840d1060 msgr2=0x7f4f8413bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.191+0000 7f4f82800640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f840d1060 0x7f4f8413bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.191+0000 7f4f82800640 1 -- 172.21.15.131:0/2557895919 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f841401d0 con 0x7f4f84072b70 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.191+0000 7f4f82800640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f84072b70 0x7f4f8414b2c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400ba80 tx=0x7f4f7400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.192+0000 7f4f7b400640 1 -- 172.21.15.131:0/2557895919 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7400cc00 con 0x7f4f84072b70 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.192+0000 7f4f7b400640 1 -- 172.21.15.131:0/2557895919 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f7400cda0 con 0x7f4f84072b70 2024-04-28T12:38:02.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.192+0000 7f4f7b400640 1 -- 172.21.15.131:0/2557895919 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74003bb0 con 0x7f4f84072b70 2024-04-28T12:38:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.192+0000 7f4f89400640 1 -- 172.21.15.131:0/2557895919 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f841404b0 con 0x7f4f84072b70 2024-04-28T12:38:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.192+0000 7f4f89400640 1 -- 172.21.15.131:0/2557895919 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f84140a70 con 0x7f4f84072b70 2024-04-28T12:38:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.193+0000 7f4f89400640 1 -- 172.21.15.131:0/2557895919 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f84069350 con 0x7f4f84072b70 2024-04-28T12:38:02.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.194+0000 7f4f7b400640 1 -- 172.21.15.131:0/2557895919 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f74014440 con 0x7f4f84072b70 2024-04-28T12:38:02.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.195+0000 7f4f7b400640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4f6006c8f0 0x7f4f6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:02.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.195+0000 7f4f7b400640 1 -- 172.21.15.131:0/2557895919 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4f7408be80 con 0x7f4f84072b70 2024-04-28T12:38:02.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.195+0000 7f4f79e00640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4f6006c8f0 0x7f4f6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:02.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.196+0000 7f4f79e00640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4f6006c8f0 0x7f4f6006edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c006fd0 tx=0x7f4f6c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:02.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.200+0000 7f4f7b400640 1 -- 172.21.15.131:0/2557895919 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f74059ed0 con 0x7f4f84072b70 2024-04-28T12:38:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.438+0000 7f4f89400640 1 -- 172.21.15.131:0/2557895919 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f840b1b40 con 0x7f4f84072b70 2024-04-28T12:38:02.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.440+0000 7f4f7b400640 1 -- 172.21.15.131:0/2557895919 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5191 (secure 0 0 0) 0x7f4f740597d0 con 0x7f4f84072b70 2024-04-28T12:38:02.436 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:02.437 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1064054651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1064054651},{"type":"v1","addr":"172.21.15.131:6834","nonce":1064054651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/398655652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":398655652},{"type":"v1","addr":"172.21.15.131:6835","nonce":398655652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2619661531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2619661531},{"type":"v1","addr":"172.21.15.157:6837","nonce":2619661531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:37:22.236351+0000","modified":"2024-04-28T12:37:50.533095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1937780695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1937780695},{"type":"v1","addr":"172.21.15.157:6835","nonce":1937780695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34160,"qdb_cluster":[34160]},"id":1}]} 2024-04-28T12:38:02.437 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-04-28T12:38:02.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.442+0000 7f4f89400640 1 -- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4f6006c8f0 msgr2=0x7f4f6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:02.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.442+0000 7f4f89400640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4f6006c8f0 0x7f4f6006edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c006fd0 tx=0x7f4f6c008040 comp rx=0 tx=0).stop 2024-04-28T12:38:02.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.442+0000 7f4f89400640 1 -- 172.21.15.131:0/2557895919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f84072b70 msgr2=0x7f4f8414b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:02.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.442+0000 7f4f89400640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f84072b70 0x7f4f8414b2c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400ba80 tx=0x7f4f7400bf40 comp rx=0 tx=0).stop 2024-04-28T12:38:02.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.442+0000 7f4f89400640 1 -- 172.21.15.131:0/2557895919 shutdown_connections 2024-04-28T12:38:02.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.442+0000 7f4f89400640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4f6006c8f0 0x7f4f6006edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.442+0000 7f4f89400640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f841467c0 0x7f4f8413c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.442+0000 7f4f89400640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f840d1060 0x7f4f8413bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.442+0000 7f4f89400640 1 --2- 172.21.15.131:0/2557895919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f84072b70 0x7f4f8414b2c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.442+0000 7f4f89400640 1 -- 172.21.15.131:0/2557895919 >> 172.21.15.131:0/2557895919 conn(0x7f4f840b2900 msgr2=0x7f4f840b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:02.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.442+0000 7f4f89400640 1 -- 172.21.15.131:0/2557895919 shutdown_connections 2024-04-28T12:38:02.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.443+0000 7f4f89400640 1 -- 172.21.15.131:0/2557895919 wait complete. 2024-04-28T12:38:02.457 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:02.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.654+0000 7fe4bb200640 1 Processor -- start 2024-04-28T12:38:02.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.654+0000 7fe4bb200640 1 -- start start 2024-04-28T12:38:02.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.656+0000 7fe4bb200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe4bc072620 0x7fe4bc0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:02.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.656+0000 7fe4bb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4bc0ce2b0 0x7fe4bc06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:02.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.656+0000 7fe4bb200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4bc072a80 con 0x7fe4bc0cd180 2024-04-28T12:38:02.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.656+0000 7fe4bb200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe4bc0ce720 con 0x7fe4bc072620 2024-04-28T12:38:02.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.656+0000 7fe4bb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe4bc0cd560 con 0x7fe4bc0ce2b0 2024-04-28T12:38:02.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.656+0000 7fe4b9400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe4bc072620 0x7fe4bc0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:02.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.656+0000 7fe4b9400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe4bc072620 0x7fe4bc0cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42886/0 (socket says 172.21.15.131:42886) 2024-04-28T12:38:02.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.656+0000 7fe4b9400640 1 -- 172.21.15.131:0/1783071887 learned_addr learned my addr 172.21.15.131:0/1783071887 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.656+0000 7fe4ba800640 1 --2- 172.21.15.131:0/1783071887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4bc0ce2b0 0x7fe4bc06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.656+0000 7fe4b9400640 1 -- 172.21.15.131:0/1783071887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4bc0ce2b0 msgr2=0x7fe4bc06dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.657+0000 7fe4b9400640 1 --2- 172.21.15.131:0/1783071887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4bc0ce2b0 0x7fe4bc06dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.657+0000 7fe4b9400640 1 -- 172.21.15.131:0/1783071887 >> v1:172.21.15.131:6789/0 conn(0x7fe4bc0cd180 legacy=0x7fe4bc15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.657+0000 7fe4b9400640 1 -- 172.21.15.131:0/1783071887 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4bc072d00 con 0x7fe4bc072620 2024-04-28T12:38:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.657+0000 7fe4ba800640 1 --2- 172.21.15.131:0/1783071887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4bc0ce2b0 0x7fe4bc06dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.658+0000 7fe4b9400640 1 --2- 172.21.15.131:0/1783071887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4bc072620 0x7fe4bc0cdd70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009870 tx=0x7fe4a402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31eda3dc098ba81b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.658+0000 7fe4b8a00640 1 -- 172.21.15.131:0/1783071887 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4037070 con 0x7fe4bc072620 2024-04-28T12:38:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.658+0000 7fe4b8a00640 1 -- 172.21.15.131:0/1783071887 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a4032440 con 0x7fe4bc072620 2024-04-28T12:38:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.658+0000 7fe4b8a00640 1 -- 172.21.15.131:0/1783071887 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4032810 con 0x7fe4bc072620 2024-04-28T12:38:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.658+0000 7fe4bb200640 1 -- 172.21.15.131:0/1783071887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4bc072620 msgr2=0x7fe4bc0cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.658+0000 7fe4bb200640 1 --2- 172.21.15.131:0/1783071887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4bc072620 0x7fe4bc0cdd70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009870 tx=0x7fe4a402a250 comp rx=0 tx=0).stop 2024-04-28T12:38:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.659+0000 7fe4bb200640 1 -- 172.21.15.131:0/1783071887 shutdown_connections 2024-04-28T12:38:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.659+0000 7fe4bb200640 1 --2- 172.21.15.131:0/1783071887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4bc0ce2b0 0x7fe4bc06dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.659+0000 7fe4bb200640 1 --2- 172.21.15.131:0/1783071887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4bc072620 0x7fe4bc0cdd70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.659+0000 7fe4bb200640 1 -- 172.21.15.131:0/1783071887 >> 172.21.15.131:0/1783071887 conn(0x7fe4bc068090 msgr2=0x7fe4bc068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.659+0000 7fe4bb200640 1 -- 172.21.15.131:0/1783071887 shutdown_connections 2024-04-28T12:38:02.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.659+0000 7fe4bb200640 1 -- 172.21.15.131:0/1783071887 wait complete. 2024-04-28T12:38:02.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.660+0000 7fe4bb200640 1 Processor -- start 2024-04-28T12:38:02.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.660+0000 7fe4bb200640 1 -- start start 2024-04-28T12:38:02.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.661+0000 7fe4bb200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4bc072620 0x7fe4bc1450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:02.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.661+0000 7fe4bb200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4bc0cd180 0x7fe4bc15c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:02.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.661+0000 7fe4bb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4bc0ce2b0 0x7fe4bc143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:02.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.661+0000 7fe4bb200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc0735a0 con 0x7fe4bc0cd180 2024-04-28T12:38:02.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.661+0000 7fe4bb200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc0732c0 con 0x7fe4bc072620 2024-04-28T12:38:02.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.661+0000 7fe4bb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe4bc073da0 con 0x7fe4bc0ce2b0 2024-04-28T12:38:02.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.662+0000 7fe4b9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4bc0cd180 0x7fe4bc15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:02.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.662+0000 7fe4b9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4bc0cd180 0x7fe4bc15c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42898/0 (socket says 172.21.15.131:42898) 2024-04-28T12:38:02.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.662+0000 7fe4b9400640 1 -- 172.21.15.131:0/2924620053 learned_addr learned my addr 172.21.15.131:0/2924620053 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:02.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.662+0000 7fe4b9400640 1 -- 172.21.15.131:0/2924620053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4bc0ce2b0 msgr2=0x7fe4bc143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:02.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.662+0000 7fe4b9400640 1 --2- 172.21.15.131:0/2924620053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4bc0ce2b0 0x7fe4bc143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.662+0000 7fe4b9400640 1 -- 172.21.15.131:0/2924620053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4bc072620 msgr2=0x7fe4bc1450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:02.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.662+0000 7fe4b9400640 1 --2- 172.21.15.131:0/2924620053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4bc072620 0x7fe4bc1450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.662+0000 7fe4b9400640 1 -- 172.21.15.131:0/2924620053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4bc144040 con 0x7fe4bc0cd180 2024-04-28T12:38:02.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.662+0000 7fe4b9400640 1 --2- 172.21.15.131:0/2924620053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4bc0cd180 0x7fe4bc15c330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4000c00 tx=0x7fe4a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.663+0000 7fe4b2000640 1 -- 172.21.15.131:0/2924620053 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4037040 con 0x7fe4bc0cd180 2024-04-28T12:38:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.663+0000 7fe4b2000640 1 -- 172.21.15.131:0/2924620053 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a4042070 con 0x7fe4bc0cd180 2024-04-28T12:38:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.663+0000 7fe4b2000640 1 -- 172.21.15.131:0/2924620053 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4030bd0 con 0x7fe4bc0cd180 2024-04-28T12:38:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.663+0000 7fe4bb200640 1 -- 172.21.15.131:0/2924620053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4bc144320 con 0x7fe4bc0cd180 2024-04-28T12:38:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.663+0000 7fe4bb200640 1 -- 172.21.15.131:0/2924620053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4bc1448e0 con 0x7fe4bc0cd180 2024-04-28T12:38:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.664+0000 7fe4bb200640 1 -- 172.21.15.131:0/2924620053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4bc069350 con 0x7fe4bc0cd180 2024-04-28T12:38:02.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.665+0000 7fe4b2000640 1 -- 172.21.15.131:0/2924620053 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4a4047460 con 0x7fe4bc0cd180 2024-04-28T12:38:02.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.666+0000 7fe4b2000640 1 --2- 172.21.15.131:0/2924620053 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe49806c920 0x7fe49806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:02.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.666+0000 7fe4b2000640 1 -- 172.21.15.131:0/2924620053 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe4a40acbb0 con 0x7fe4bc0cd180 2024-04-28T12:38:02.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.666+0000 7fe4b9e00640 1 --2- 172.21.15.131:0/2924620053 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe49806c920 0x7fe49806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:02.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.667+0000 7fe4b9e00640 1 --2- 172.21.15.131:0/2924620053 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe49806c920 0x7fe49806ede0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0012e0 tx=0x7fe4ac007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:02.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.671+0000 7fe4b2000640 1 -- 172.21.15.131:0/2924620053 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe4a407ac00 con 0x7fe4bc0cd180 2024-04-28T12:38:02.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.877+0000 7fe4bb200640 1 -- 172.21.15.131:0/2924620053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4bc0d0670 con 0x7fe4bc0cd180 2024-04-28T12:38:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.879+0000 7fe4b2000640 1 -- 172.21.15.131:0/2924620053 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18538 (secure 0 0 0) 0x7fe4a407a500 con 0x7fe4bc0cd180 2024-04-28T12:38:02.876 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:02.876 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":24,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:37:27.317547+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:02.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.881+0000 7fe4bb200640 1 -- 172.21.15.131:0/2924620053 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe49806c920 msgr2=0x7fe49806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:02.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.881+0000 7fe4bb200640 1 --2- 172.21.15.131:0/2924620053 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe49806c920 0x7fe49806ede0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0012e0 tx=0x7fe4ac007d50 comp rx=0 tx=0).stop 2024-04-28T12:38:02.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.881+0000 7fe4bb200640 1 -- 172.21.15.131:0/2924620053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4bc0cd180 msgr2=0x7fe4bc15c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:02.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.881+0000 7fe4bb200640 1 --2- 172.21.15.131:0/2924620053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4bc0cd180 0x7fe4bc15c330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4000c00 tx=0x7fe4a40046c0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.881+0000 7fe4bb200640 1 -- 172.21.15.131:0/2924620053 shutdown_connections 2024-04-28T12:38:02.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.881+0000 7fe4bb200640 1 --2- 172.21.15.131:0/2924620053 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe49806c920 0x7fe49806ede0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.881+0000 7fe4bb200640 1 --2- 172.21.15.131:0/2924620053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4bc0ce2b0 0x7fe4bc143740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.881+0000 7fe4bb200640 1 --2- 172.21.15.131:0/2924620053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4bc0cd180 0x7fe4bc15c330 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.881+0000 7fe4bb200640 1 --2- 172.21.15.131:0/2924620053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4bc072620 0x7fe4bc1450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:02.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.881+0000 7fe4bb200640 1 -- 172.21.15.131:0/2924620053 >> 172.21.15.131:0/2924620053 conn(0x7fe4bc068090 msgr2=0x7fe4bc06ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:02.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.882+0000 7fe4bb200640 1 -- 172.21.15.131:0/2924620053 shutdown_connections 2024-04-28T12:38:02.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:02.882+0000 7fe4bb200640 1 -- 172.21.15.131:0/2924620053 wait complete. 2024-04-28T12:38:02.898 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:38:03.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.102+0000 7f3c2f200640 1 Processor -- start 2024-04-28T12:38:03.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.102+0000 7f3c2f200640 1 -- start start 2024-04-28T12:38:03.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.103+0000 7f3c2f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c300af710 0x7f3c300afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.103+0000 7f3c2f200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c30072db0 con 0x7f3c300cf600 2024-04-28T12:38:03.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.103+0000 7f3c2f200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c300cee40 con 0x7f3c3015bb90 2024-04-28T12:38:03.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.103+0000 7f3c2f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3c300cec10 con 0x7f3c300af710 2024-04-28T12:38:03.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.103+0000 7f3c2e800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3c300cf600 0x7f3c300cfa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44626/0 (socket says 172.21.15.131:44626) 2024-04-28T12:38:03.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.103+0000 7f3c2e800640 1 -- 172.21.15.131:0/721910331 learned_addr learned my addr 172.21.15.131:0/721910331 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:03.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.104+0000 7f3c25400640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c300af710 0x7f3c300afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:03.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.104+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 622298993 0 0) 0x7f3c30072db0 con 0x7f3c300cf600 2024-04-28T12:38:03.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.104+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c14003520 con 0x7f3c300cf600 2024-04-28T12:38:03.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.104+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2634956605 0 0) 0x7f3c14003520 con 0x7f3c300cf600 2024-04-28T12:38:03.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.104+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c300af710 msgr2=0x7f3c300afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:03.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.104+0000 7f3c2d400640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c300af710 0x7f3c300afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.104+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 >> v1:172.21.15.157:6789/0 conn(0x7f3c3015bb90 legacy=0x7f3c300737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.104+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c300727f0 con 0x7f3c300cf600 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.104+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f3c280019f0 con 0x7f3c300cf600 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.104+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 >> v1:172.21.15.131:6789/0 conn(0x7f3c300cf600 legacy=0x7f3c300cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.104+0000 7f3c2d400640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c140039e0 0x7f3c14003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.104+0000 7f3c2d400640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c14004800 0x7f3c14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c2d400640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c300af710 0x7f3c140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1400ba70 con 0x7f3c14004800 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1400bc20 con 0x7f3c140039e0 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1400e030 con 0x7f3c300af710 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c25400640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c14004800 0x7f3c14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c2e800640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c300af710 0x7f3c140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c2de00640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c140039e0 0x7f3c14003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c25400640 1 -- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c300af710 msgr2=0x7f3c140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:03.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c25400640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c300af710 0x7f3c140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c25400640 1 -- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c140039e0 msgr2=0x7f3c14003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:03.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c25400640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c140039e0 0x7f3c14003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c25400640 1 -- 172.21.15.131:0/721910331 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c14003520 con 0x7f3c14004800 2024-04-28T12:38:03.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c2de00640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c140039e0 0x7f3c14003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:03.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.105+0000 7f3c2e800640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c300af710 0x7f3c140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:03.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.106+0000 7f3c25400640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c14004800 0x7f3c14004c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009a90 tx=0x7f3c14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4356bf0fdad974a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:03.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.106+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18036070 con 0x7f3c14004800 2024-04-28T12:38:03.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.106+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c18031400 con 0x7f3c14004800 2024-04-28T12:38:03.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.106+0000 7f3c2d400640 1 -- 172.21.15.131:0/721910331 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18009690 con 0x7f3c14004800 2024-04-28T12:38:03.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.107+0000 7f3c2f200640 1 -- 172.21.15.131:0/721910331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c14004800 msgr2=0x7f3c14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:03.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.107+0000 7f3c2f200640 1 --2- 172.21.15.131:0/721910331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c14004800 0x7f3c14004c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009a90 tx=0x7f3c14005830 comp rx=0 tx=0).stop 2024-04-28T12:38:03.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.107+0000 7f3c2e800640 1 -- 172.21.15.131:0/721910331 reap_dead start 2024-04-28T12:38:03.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.107+0000 7f3c2f200640 1 -- 172.21.15.131:0/721910331 shutdown_connections 2024-04-28T12:38:03.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.107+0000 7f3c2f200640 1 -- 172.21.15.131:0/721910331 >> 172.21.15.131:0/721910331 conn(0x7f3c30068090 msgr2=0x7f3c30068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:03.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.107+0000 7f3c2f200640 1 -- 172.21.15.131:0/721910331 shutdown_connections 2024-04-28T12:38:03.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.107+0000 7f3c2f200640 1 -- 172.21.15.131:0/721910331 wait complete. 2024-04-28T12:38:03.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.109+0000 7f3c2f200640 1 Processor -- start 2024-04-28T12:38:03.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.109+0000 7f3c2f200640 1 -- start start 2024-04-28T12:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.110+0000 7f3c2f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c14004800 0x7f3c30144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.110+0000 7f3c2f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c3015c000 0x7f3c3015ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.110+0000 7f3c2f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c3015d210 0x7f3c301f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.110+0000 7f3c2f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30073eb0 con 0x7f3c3015d210 2024-04-28T12:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.110+0000 7f3c2f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3c300725a0 con 0x7f3c3015c000 2024-04-28T12:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.110+0000 7f3c2f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3c300cf0e0 con 0x7f3c14004800 2024-04-28T12:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.110+0000 7f3c2e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c3015d210 0x7f3c301f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.110+0000 7f3c2e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c3015d210 0x7f3c301f3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42908/0 (socket says 172.21.15.131:42908) 2024-04-28T12:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.110+0000 7f3c2e800640 1 -- 172.21.15.131:0/967935713 learned_addr learned my addr 172.21.15.131:0/967935713 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.111+0000 7f3c2e800640 1 -- 172.21.15.131:0/967935713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c14004800 msgr2=0x7f3c30144140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.111+0000 7f3c2e800640 1 --2- 172.21.15.131:0/967935713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c14004800 0x7f3c30144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.111+0000 7f3c2e800640 1 -- 172.21.15.131:0/967935713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c3015c000 msgr2=0x7f3c3015ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:03.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.111+0000 7f3c2e800640 1 --2- 172.21.15.131:0/967935713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c3015c000 0x7f3c3015ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.111+0000 7f3c2e800640 1 -- 172.21.15.131:0/967935713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c30153280 con 0x7f3c3015d210 2024-04-28T12:38:03.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.111+0000 7f3c2e800640 1 --2- 172.21.15.131:0/967935713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c3015d210 0x7f3c301f3fb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3c3006f680 tx=0x7f3c2800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:03.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.111+0000 7f3c26a00640 1 -- 172.21.15.131:0/967935713 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800b870 con 0x7f3c3015d210 2024-04-28T12:38:03.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.111+0000 7f3c26a00640 1 -- 172.21.15.131:0/967935713 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c2800ba10 con 0x7f3c3015d210 2024-04-28T12:38:03.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.111+0000 7f3c26a00640 1 -- 172.21.15.131:0/967935713 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800bdc0 con 0x7f3c3015d210 2024-04-28T12:38:03.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.111+0000 7f3c2f200640 1 -- 172.21.15.131:0/967935713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c30153560 con 0x7f3c3015d210 2024-04-28T12:38:03.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.112+0000 7f3c2f200640 1 -- 172.21.15.131:0/967935713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c30153b20 con 0x7f3c3015d210 2024-04-28T12:38:03.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.113+0000 7f3c2f200640 1 -- 172.21.15.131:0/967935713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c300b10f0 con 0x7f3c3015d210 2024-04-28T12:38:03.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.114+0000 7f3c26a00640 1 -- 172.21.15.131:0/967935713 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c2801b020 con 0x7f3c3015d210 2024-04-28T12:38:03.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.115+0000 7f3c26a00640 1 --2- 172.21.15.131:0/967935713 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3c0c06c920 0x7f3c0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.115+0000 7f3c26a00640 1 -- 172.21.15.131:0/967935713 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3c2808bff0 con 0x7f3c3015d210 2024-04-28T12:38:03.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.115+0000 7f3c2de00640 1 --2- 172.21.15.131:0/967935713 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3c0c06c920 0x7f3c0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:03.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.116+0000 7f3c2de00640 1 --2- 172.21.15.131:0/967935713 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3c0c06c920 0x7f3c0c06ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3c3015c930 tx=0x7f3c20009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:03.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.119+0000 7f3c26a00640 1 -- 172.21.15.131:0/967935713 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3c2805a040 con 0x7f3c3015d210 2024-04-28T12:38:03.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.360+0000 7f3c2f200640 1 -- 172.21.15.131:0/967935713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c30069350 con 0x7f3c3015d210 2024-04-28T12:38:03.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.361+0000 7f3c26a00640 1 -- 172.21.15.131:0/967935713 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5191 (secure 0 0 0) 0x7f3c28059940 con 0x7f3c3015d210 2024-04-28T12:38:03.358 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:03.358 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1064054651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1064054651},{"type":"v1","addr":"172.21.15.131:6834","nonce":1064054651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/398655652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":398655652},{"type":"v1","addr":"172.21.15.131:6835","nonce":398655652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2619661531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2619661531},{"type":"v1","addr":"172.21.15.157:6837","nonce":2619661531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:37:22.236351+0000","modified":"2024-04-28T12:37:50.533095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1937780695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1937780695},{"type":"v1","addr":"172.21.15.157:6835","nonce":1937780695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34160,"qdb_cluster":[34160]},"id":1}]} 2024-04-28T12:38:03.358 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-04-28T12:38:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.364+0000 7f3c2f200640 1 -- 172.21.15.131:0/967935713 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3c0c06c920 msgr2=0x7f3c0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.364+0000 7f3c2f200640 1 --2- 172.21.15.131:0/967935713 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3c0c06c920 0x7f3c0c06ede0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3c3015c930 tx=0x7f3c20009040 comp rx=0 tx=0).stop 2024-04-28T12:38:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.364+0000 7f3c2f200640 1 -- 172.21.15.131:0/967935713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c3015d210 msgr2=0x7f3c301f3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.364+0000 7f3c2f200640 1 --2- 172.21.15.131:0/967935713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c3015d210 0x7f3c301f3fb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3c3006f680 tx=0x7f3c2800b0d0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.364+0000 7f3c2f200640 1 -- 172.21.15.131:0/967935713 shutdown_connections 2024-04-28T12:38:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.364+0000 7f3c2f200640 1 --2- 172.21.15.131:0/967935713 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3c0c06c920 0x7f3c0c06ede0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.364+0000 7f3c2f200640 1 --2- 172.21.15.131:0/967935713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c3015d210 0x7f3c301f3fb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.364+0000 7f3c2f200640 1 --2- 172.21.15.131:0/967935713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c3015c000 0x7f3c3015ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.364+0000 7f3c2f200640 1 --2- 172.21.15.131:0/967935713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c14004800 0x7f3c30144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.364+0000 7f3c2f200640 1 -- 172.21.15.131:0/967935713 >> 172.21.15.131:0/967935713 conn(0x7f3c30068090 msgr2=0x7f3c30068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:03.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.364+0000 7f3c2f200640 1 -- 172.21.15.131:0/967935713 shutdown_connections 2024-04-28T12:38:03.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.364+0000 7f3c2f200640 1 -- 172.21.15.131:0/967935713 wait complete. 2024-04-28T12:38:03.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-28T12:37:22.236351+0000', 'modified': '2024-04-28T12:37:50.533095+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34160}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34160': {'gid': 34160, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/1937780695', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 1937780695}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 1937780695}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34160, 'qdb_cluster': [34160]} 2024-04-28T12:38:03.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-28T12:38:03.783 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-28T12:38:03.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.989+0000 7f5756e00640 1 Processor -- start 2024-04-28T12:38:03.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.990+0000 7f5756e00640 1 -- start start 2024-04-28T12:38:03.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.991+0000 7f5756e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f575015eef0 0x7f575015f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.991+0000 7f5756e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57500c9080 0x7f57500d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.991+0000 7f5756e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57500d3a30 con 0x7f57500d05b0 2024-04-28T12:38:03.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.991+0000 7f5756e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f57500cfad0 con 0x7f575015eef0 2024-04-28T12:38:03.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.991+0000 7f5756e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f57500cfdb0 con 0x7f57500c9080 2024-04-28T12:38:03.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.991+0000 7f574fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f575015eef0 0x7f575015f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:03.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.991+0000 7f574fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f575015eef0 0x7f575015f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42920/0 (socket says 172.21.15.131:42920) 2024-04-28T12:38:03.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.991+0000 7f574fe00640 1 -- 172.21.15.131:0/81583546 learned_addr learned my addr 172.21.15.131:0/81583546 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:03.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.992+0000 7f5754e00640 1 --2- 172.21.15.131:0/81583546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57500c9080 0x7f57500d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:03.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.992+0000 7f574fe00640 1 -- 172.21.15.131:0/81583546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57500c9080 msgr2=0x7f57500d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:03.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.992+0000 7f574fe00640 1 --2- 172.21.15.131:0/81583546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57500c9080 0x7f57500d4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.992+0000 7f574fe00640 1 -- 172.21.15.131:0/81583546 >> v1:172.21.15.157:6789/0 conn(0x7f57500d05b0 legacy=0x7f57500c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:03.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.992+0000 7f574fe00640 1 -- 172.21.15.131:0/81583546 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57500d0090 con 0x7f575015eef0 2024-04-28T12:38:03.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.993+0000 7f574fe00640 1 --2- 172.21.15.131:0/81583546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f575015eef0 0x7f575015f2d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5740009870 tx=0x7f574002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8285a5281e5b126 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:03.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.993+0000 7f574ea00640 1 -- 172.21.15.131:0/81583546 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740037070 con 0x7f575015eef0 2024-04-28T12:38:03.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.993+0000 7f574ea00640 1 -- 172.21.15.131:0/81583546 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5740032440 con 0x7f575015eef0 2024-04-28T12:38:03.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.993+0000 7f574ea00640 1 -- 172.21.15.131:0/81583546 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740032810 con 0x7f575015eef0 2024-04-28T12:38:03.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.994+0000 7f5756e00640 1 -- 172.21.15.131:0/81583546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f575015eef0 msgr2=0x7f575015f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:03.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.994+0000 7f5756e00640 1 --2- 172.21.15.131:0/81583546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f575015eef0 0x7f575015f2d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5740009870 tx=0x7f574002a180 comp rx=0 tx=0).stop 2024-04-28T12:38:03.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.994+0000 7f5756e00640 1 -- 172.21.15.131:0/81583546 shutdown_connections 2024-04-28T12:38:03.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.994+0000 7f5756e00640 1 --2- 172.21.15.131:0/81583546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f57500c9080 0x7f57500d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.994+0000 7f5756e00640 1 --2- 172.21.15.131:0/81583546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f575015eef0 0x7f575015f2d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.994+0000 7f5756e00640 1 -- 172.21.15.131:0/81583546 >> 172.21.15.131:0/81583546 conn(0x7f575008e7e0 msgr2=0x7f575009e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:03.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.994+0000 7f5756e00640 1 -- 172.21.15.131:0/81583546 shutdown_connections 2024-04-28T12:38:03.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.994+0000 7f5756e00640 1 -- 172.21.15.131:0/81583546 wait complete. 2024-04-28T12:38:03.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.995+0000 7f5756e00640 1 Processor -- start 2024-04-28T12:38:03.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.996+0000 7f5756e00640 1 -- start start 2024-04-28T12:38:03.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5756e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57500d05b0 0x7f57500b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5756e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f575015eef0 0x7f575014e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5756e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57500b5370 0x7f57500b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5756e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f57500d0be0 con 0x7f57500b5370 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5756e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f57500d0a60 con 0x7f57500d05b0 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5756e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f57500d0d60 con 0x7f575015eef0 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5754e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57500b5370 0x7f57500b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5754e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57500b5370 0x7f57500b7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42924/0 (socket says 172.21.15.131:42924) 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5754e00640 1 -- 172.21.15.131:0/199793845 learned_addr learned my addr 172.21.15.131:0/199793845 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f574f400640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f575015eef0 0x7f575014e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f574fe00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57500d05b0 0x7f57500b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5754e00640 1 -- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f575015eef0 msgr2=0x7f575014e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5754e00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f575015eef0 0x7f575014e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5754e00640 1 -- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57500d05b0 msgr2=0x7f57500b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:03.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5754e00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57500d05b0 0x7f57500b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:03.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f5754e00640 1 -- 172.21.15.131:0/199793845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57500b42d0 con 0x7f57500b5370 2024-04-28T12:38:03.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.997+0000 7f574f400640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f575015eef0 0x7f575014e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:03.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.998+0000 7f5754e00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57500b5370 0x7f57500b7df0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5748007e60 tx=0x7f574800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:03.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.998+0000 7f5747e00640 1 -- 172.21.15.131:0/199793845 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748014620 con 0x7f57500b5370 2024-04-28T12:38:03.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.998+0000 7f5747e00640 1 -- 172.21.15.131:0/199793845 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5748019070 con 0x7f57500b5370 2024-04-28T12:38:03.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.998+0000 7f5747e00640 1 -- 172.21.15.131:0/199793845 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748014a00 con 0x7f57500b5370 2024-04-28T12:38:03.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.998+0000 7f5756e00640 1 -- 172.21.15.131:0/199793845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57500b45b0 con 0x7f57500b5370 2024-04-28T12:38:03.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:03.998+0000 7f5756e00640 1 -- 172.21.15.131:0/199793845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57500b4b70 con 0x7f57500b5370 2024-04-28T12:38:03.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.001+0000 7f5747e00640 1 -- 172.21.15.131:0/199793845 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5748014ba0 con 0x7f57500b5370 2024-04-28T12:38:03.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.002+0000 7f5747e00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f573006c8f0 0x7f573006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:03.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.002+0000 7f5747e00640 1 -- 172.21.15.131:0/199793845 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f574808c8a0 con 0x7f57500b5370 2024-04-28T12:38:03.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.002+0000 7f5759bdb740 1 Processor -- start 2024-04-28T12:38:03.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.002+0000 7f574fe00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f573006c8f0 0x7f573006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:03.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.003+0000 7f5759bdb740 1 -- start start 2024-04-28T12:38:04.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.003+0000 7f5759bdb740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff115e0 0x55a43ff13740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.003+0000 7f5759bdb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff4dcb0 0x55a43ff119c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.003+0000 7f5759bdb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 0x55a43ff3c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.003+0000 7f5759bdb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a43fe75600 con 0x55a43ff3c0a0 2024-04-28T12:38:04.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.003+0000 7f5759bdb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a43fe12100 con 0x55a43ff4dcb0 2024-04-28T12:38:04.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.003+0000 7f5759bdb740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55a43fe715c0 con 0x55a43ff115e0 2024-04-28T12:38:04.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f574fe00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f573006c8f0 0x7f573006edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5740034000 tx=0x7f574002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f5754e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 0x55a43ff3c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f5754e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 0x55a43ff3c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42930/0 (socket says 172.21.15.131:42930) 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f5754e00640 1 -- 172.21.15.131:0/913190656 learned_addr learned my addr 172.21.15.131:0/913190656 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f574fe00640 1 --2- 172.21.15.131:0/913190656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff115e0 0x55a43ff13740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f574f400640 1 --2- 172.21.15.131:0/913190656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff4dcb0 0x55a43ff119c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f5754e00640 1 -- 172.21.15.131:0/913190656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff115e0 msgr2=0x55a43ff13740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f5754e00640 1 --2- 172.21.15.131:0/913190656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff115e0 0x55a43ff13740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f5754e00640 1 -- 172.21.15.131:0/913190656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff4dcb0 msgr2=0x55a43ff119c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f5754e00640 1 --2- 172.21.15.131:0/913190656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff4dcb0 0x55a43ff119c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f5754e00640 1 -- 172.21.15.131:0/913190656 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43fb872d0 con 0x55a43ff3c0a0 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f574fe00640 1 --2- 172.21.15.131:0/913190656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff115e0 0x55a43ff13740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f574f400640 1 --2- 172.21.15.131:0/913190656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff4dcb0 0x55a43ff119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.004+0000 7f5754e00640 1 --2- 172.21.15.131:0/913190656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 0x55a43ff3c520 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f57480628a0 tx=0x7f574805a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a060ab59d50632 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.005+0000 7f5744c00640 1 -- 172.21.15.131:0/913190656 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748072020 con 0x55a43ff3c0a0 2024-04-28T12:38:04.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.005+0000 7f5744c00640 1 -- 172.21.15.131:0/913190656 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f574808ae50 con 0x55a43ff3c0a0 2024-04-28T12:38:04.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.005+0000 7f5744c00640 1 -- 172.21.15.131:0/913190656 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574805e510 con 0x55a43ff3c0a0 2024-04-28T12:38:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.005+0000 7f5759bdb740 1 -- 172.21.15.131:0/913190656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 msgr2=0x55a43ff3c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.005+0000 7f5759bdb740 1 --2- 172.21.15.131:0/913190656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 0x55a43ff3c520 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f57480628a0 tx=0x7f574805a860 comp rx=0 tx=0).stop 2024-04-28T12:38:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.005+0000 7f5759bdb740 1 -- 172.21.15.131:0/913190656 shutdown_connections 2024-04-28T12:38:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.005+0000 7f5759bdb740 1 --2- 172.21.15.131:0/913190656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 0x55a43ff3c520 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.005+0000 7f5759bdb740 1 --2- 172.21.15.131:0/913190656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff4dcb0 0x55a43ff119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.005+0000 7f5759bdb740 1 --2- 172.21.15.131:0/913190656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff115e0 0x55a43ff13740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.005+0000 7f5759bdb740 1 -- 172.21.15.131:0/913190656 >> 172.21.15.131:0/913190656 conn(0x55a43fea5d10 msgr2=0x55a43ff570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.006+0000 7f5759bdb740 1 -- 172.21.15.131:0/913190656 shutdown_connections 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.006+0000 7f5759bdb740 1 -- 172.21.15.131:0/913190656 wait complete. 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.006+0000 7f5759bdb740 1 -- start start 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.006+0000 7f5759bdb740 1 Processor -- start 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.007+0000 7f5759bdb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff115e0 0x55a43ff3d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.007+0000 7f5759bdb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 0x55a43ff51f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.007+0000 7f5759bdb740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff4dcb0 0x55a43ff382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.007+0000 7f5759bdb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a43fdfcf70 con 0x55a43ff3c0a0 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.007+0000 7f5759bdb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a43ff39bf0 con 0x55a43ff115e0 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.007+0000 7f5759bdb740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55a43ff68600 con 0x55a43ff4dcb0 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f5759bdb740 10 client.0 _ll_register_callbacks cb 0x55a43fe6b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f574f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 0x55a43ff51f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f574f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 0x55a43ff51f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42940/0 (socket says 172.21.15.131:42940) 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f574f400640 1 -- 172.21.15.131:0/781448615 learned_addr learned my addr 172.21.15.131:0/781448615 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f574f400640 1 -- 172.21.15.131:0/781448615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff4dcb0 msgr2=0x55a43ff382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f574fe00640 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff115e0 0x55a43ff3d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f574f400640 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff4dcb0 0x55a43ff382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f574f400640 1 -- 172.21.15.131:0/781448615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff115e0 msgr2=0x55a43ff3d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f574f400640 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff115e0 0x55a43ff3d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f574f400640 1 -- 172.21.15.131:0/781448615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43fc516b0 con 0x55a43ff3c0a0 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f574fe00640 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff115e0 0x55a43ff3d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f574f400640 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 0x55a43ff51f10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5750162360 tx=0x7f573c00e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cdd50ea1896d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.008+0000 7f5739600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.009+0000 7f5739600640 1 -- 172.21.15.131:0/781448615 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c007230 con 0x55a43ff3c0a0 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.009+0000 7f5739600640 1 -- 172.21.15.131:0/781448615 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f573c0073d0 con 0x55a43ff3c0a0 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.009+0000 7f5759bdb740 1 -- 172.21.15.131:0/781448615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f574808ae50 con 0x55a43ff3c0a0 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.009+0000 7f5739600640 1 -- 172.21.15.131:0/781448615 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f573c020050 con 0x55a43ff3c0a0 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.009+0000 7f5759bdb740 10 client.4779 fetch_fsmap learned FSMap version 9 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.009+0000 7f5759bdb740 1 -- 172.21.15.131:0/781448615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55a43ff38810 con 0x55a43ff3c0a0 2024-04-28T12:38:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.009+0000 7f5739600640 1 -- 172.21.15.131:0/781448615 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f574808ae50 con 0x55a43ff3c0a0 2024-04-28T12:38:04.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.009+0000 7f5739600640 1 -- 172.21.15.131:0/781448615 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f573c023020 con 0x55a43ff3c0a0 2024-04-28T12:38:04.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.009+0000 7f5759bdb740 10 client.4779 fetch_fsmap finished waiting for FSMap version 9 2024-04-28T12:38:04.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.009+0000 7f5759bdb740 10 client.4779 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T12:38:04.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.009+0000 7f5759bdb740 20 client.4779 populate_metadata read hostname 'smithi131' 2024-04-28T12:38:04.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.009+0000 7f5759bdb740 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x55a43ff3c730 0x55a43ff676f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.010+0000 7f5759bdb740 4 client.4779 mds_command: new command op to 34160 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:38:04.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.010+0000 7f5759bdb740 1 -- 172.21.15.131:0/781448615 --> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a43fe83210 con 0x55a43ff3c730 2024-04-28T12:38:04.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.010+0000 7f574fe00640 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x55a43ff3c730 0x55a43ff676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.011+0000 7f574fe00640 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x55a43ff3c730 0x55a43ff676f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b159e288463b2b server_cookie=c74bd055678c285 in_seq=0 out_seq=0 2024-04-28T12:38:04.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.011+0000 7f5739600640 10 client.4779 ms_handle_connect on v2:172.21.15.157:6834/1937780695 2024-04-28T12:38:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5739600640 1 -- 172.21.15.131:0/781448615 <== mds.0 v2:172.21.15.157:6834/1937780695 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a43fe83210 con 0x55a43ff3c730 2024-04-28T12:38:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5739600640 10 client.4779 handle_command_reply: tid=0 2024-04-28T12:38:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5759bdb740 1 client.4779 shutdown 2024-04-28T12:38:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5759bdb740 1 -- 172.21.15.131:0/781448615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 msgr2=0x55a43ff51f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5759bdb740 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 0x55a43ff51f10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5750162360 tx=0x7f573c00e590 comp rx=0 tx=0).stop 2024-04-28T12:38:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5759bdb740 1 -- 172.21.15.131:0/781448615 shutdown_connections 2024-04-28T12:38:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5759bdb740 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x55a43ff3c730 0x55a43ff676f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5759bdb740 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff4dcb0 0x55a43ff382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5759bdb740 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff3c0a0 0x55a43ff51f10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5759bdb740 1 --2- 172.21.15.131:0/781448615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff115e0 0x55a43ff3d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5759bdb740 1 -- 172.21.15.131:0/781448615 >> 172.21.15.131:0/781448615 conn(0x55a43fea5d10 msgr2=0x55a43ff39dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5739600640 0 client.4779 ms_handle_reset on v2:172.21.15.157:6834/1937780695 2024-04-28T12:38:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5759bdb740 1 -- 172.21.15.131:0/781448615 shutdown_connections 2024-04-28T12:38:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5759bdb740 1 -- 172.21.15.131:0/781448615 wait complete. 2024-04-28T12:38:04.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.014+0000 7f5759bdb740 20 client.4779 trim_cache size 0 max 16384 2024-04-28T12:38:04.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.034+0000 7f5759bdb740 1 Processor -- start 2024-04-28T12:38:04.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.034+0000 7f5759bdb740 1 -- start start 2024-04-28T12:38:04.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.035+0000 7f5759bdb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff115e0 0x55a43ff12640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.035+0000 7f5759bdb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff3c0a0 0x55a43ff3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.035+0000 7f5759bdb740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff4dcb0 0x55a43ff67670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.035+0000 7f5759bdb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5750201c90 con 0x55a43ff115e0 2024-04-28T12:38:04.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.035+0000 7f5759bdb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5750201120 con 0x55a43ff3c0a0 2024-04-28T12:38:04.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.035+0000 7f5759bdb740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5750201640 con 0x55a43ff4dcb0 2024-04-28T12:38:04.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.035+0000 7f574fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff115e0 0x55a43ff12640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.035+0000 7f574fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff115e0 0x55a43ff12640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42954/0 (socket says 172.21.15.131:42954) 2024-04-28T12:38:04.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.035+0000 7f574fe00640 1 -- 172.21.15.131:0/3732135163 learned_addr learned my addr 172.21.15.131:0/3732135163 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:04.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.036+0000 7f574fe00640 1 -- 172.21.15.131:0/3732135163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff4dcb0 msgr2=0x55a43ff67670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:04.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.036+0000 7f574fe00640 1 --2- 172.21.15.131:0/3732135163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff4dcb0 0x55a43ff67670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.036+0000 7f574fe00640 1 -- 172.21.15.131:0/3732135163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff3c0a0 msgr2=0x55a43ff3a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:04.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.036+0000 7f574fe00640 1 --2- 172.21.15.131:0/3732135163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff3c0a0 0x55a43ff3a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.036+0000 7f574fe00640 1 -- 172.21.15.131:0/3732135163 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43fe3bb30 con 0x55a43ff115e0 2024-04-28T12:38:04.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.036+0000 7f574fe00640 1 --2- 172.21.15.131:0/3732135163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff115e0 0x55a43ff12640 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f57400090b0 tx=0x7f574002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392b989bd5b2af09 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.036+0000 7f573be00640 1 -- 172.21.15.131:0/3732135163 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740051070 con 0x55a43ff115e0 2024-04-28T12:38:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.036+0000 7f573be00640 1 -- 172.21.15.131:0/3732135163 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f574004c330 con 0x55a43ff115e0 2024-04-28T12:38:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.036+0000 7f573be00640 1 -- 172.21.15.131:0/3732135163 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740037040 con 0x55a43ff115e0 2024-04-28T12:38:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.037+0000 7f5759bdb740 1 -- 172.21.15.131:0/3732135163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff115e0 msgr2=0x55a43ff12640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.037+0000 7f5759bdb740 1 --2- 172.21.15.131:0/3732135163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff115e0 0x55a43ff12640 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f57400090b0 tx=0x7f574002c000 comp rx=0 tx=0).stop 2024-04-28T12:38:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.037+0000 7f5759bdb740 1 -- 172.21.15.131:0/3732135163 shutdown_connections 2024-04-28T12:38:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.037+0000 7f5759bdb740 1 --2- 172.21.15.131:0/3732135163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff4dcb0 0x55a43ff67670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.037+0000 7f5759bdb740 1 --2- 172.21.15.131:0/3732135163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff3c0a0 0x55a43ff3a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.037+0000 7f5759bdb740 1 --2- 172.21.15.131:0/3732135163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff115e0 0x55a43ff12640 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.037+0000 7f5759bdb740 1 -- 172.21.15.131:0/3732135163 >> 172.21.15.131:0/3732135163 conn(0x55a43fea5d10 msgr2=0x55a43ff52ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.037+0000 7f5759bdb740 1 -- 172.21.15.131:0/3732135163 shutdown_connections 2024-04-28T12:38:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.037+0000 7f5759bdb740 1 -- 172.21.15.131:0/3732135163 wait complete. 2024-04-28T12:38:04.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.038+0000 7f5759bdb740 1 -- start start 2024-04-28T12:38:04.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.038+0000 7f5759bdb740 1 Processor -- start 2024-04-28T12:38:04.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5759bdb740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff115e0 0x55a43ffba9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5759bdb740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff3c0a0 0x55a43ff4ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5759bdb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff4dcb0 0x55a43ff3f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5759bdb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a43ff3c5b0 con 0x55a43ff4dcb0 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5759bdb740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a43ff38b50 con 0x55a43ff115e0 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5759bdb740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55a43ff3cbb0 con 0x55a43ff3c0a0 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5759bdb740 10 client.0 _ll_register_callbacks cb 0x7f573c007d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5754e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff4dcb0 0x55a43ff3f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5754e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff4dcb0 0x55a43ff3f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42966/0 (socket says 172.21.15.131:42966) 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5754e00640 1 -- 172.21.15.131:0/1724036833 learned_addr learned my addr 172.21.15.131:0/1724036833 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f574fe00640 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff115e0 0x55a43ffba9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f574f400640 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff3c0a0 0x55a43ff4ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5754e00640 1 -- 172.21.15.131:0/1724036833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff3c0a0 msgr2=0x55a43ff4ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5754e00640 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff3c0a0 0x55a43ff4ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5754e00640 1 -- 172.21.15.131:0/1724036833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff115e0 msgr2=0x55a43ffba9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.039+0000 7f5754e00640 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff115e0 0x55a43ffba9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.040+0000 7f5754e00640 1 -- 172.21.15.131:0/1724036833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43ff51370 con 0x55a43ff4dcb0 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.040+0000 7f5754e00640 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff4dcb0 0x55a43ff3f150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57480565c0 tx=0x7f574808a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cf4d0d825ad87c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.040+0000 7f5739600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.040+0000 7f5739600640 1 -- 172.21.15.131:0/1724036833 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748072040 con 0x55a43ff4dcb0 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.040+0000 7f5739600640 1 -- 172.21.15.131:0/1724036833 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f57480567d0 con 0x55a43ff4dcb0 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.040+0000 7f5759bdb740 1 -- 172.21.15.131:0/1724036833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f574004c330 con 0x55a43ff4dcb0 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.040+0000 7f5739600640 1 -- 172.21.15.131:0/1724036833 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f5748081050 con 0x55a43ff4dcb0 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.040+0000 7f5759bdb740 10 client.4785 fetch_fsmap learned FSMap version 9 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.040+0000 7f5759bdb740 1 -- 172.21.15.131:0/1724036833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55a43ff4d260 con 0x55a43ff4dcb0 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.040+0000 7f5739600640 1 -- 172.21.15.131:0/1724036833 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f574004c330 con 0x55a43ff4dcb0 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.041+0000 7f5739600640 1 -- 172.21.15.131:0/1724036833 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f57480595b0 con 0x55a43ff4dcb0 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.041+0000 7f5759bdb740 10 client.4785 fetch_fsmap finished waiting for FSMap version 9 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.041+0000 7f5759bdb740 10 client.4785 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.041+0000 7f5759bdb740 20 client.4785 populate_metadata read hostname 'smithi131' 2024-04-28T12:38:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.041+0000 7f5759bdb740 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x55a43ff79920 0x55a43ff79d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.041+0000 7f5759bdb740 4 client.4785 mds_command: new command op to 34160 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:38:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.041+0000 7f5759bdb740 1 -- 172.21.15.131:0/1724036833 --> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] -- command(tid 0: {"prefix": "status"}) -- 0x55a43fe7d080 con 0x55a43ff79920 2024-04-28T12:38:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.041+0000 7f574fe00640 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x55a43ff79920 0x55a43ff79d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.042+0000 7f574fe00640 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x55a43ff79920 0x55a43ff79d00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3eb950def61b3bb server_cookie=66a7af8f7b50df86 in_seq=0 out_seq=0 2024-04-28T12:38:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.042+0000 7f5739600640 10 client.4785 ms_handle_connect on v2:172.21.15.157:6834/1937780695 2024-04-28T12:38:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5739600640 1 -- 172.21.15.131:0/1724036833 <== mds.0 v2:172.21.15.157:6834/1937780695 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a43fe7d080 con 0x55a43ff79920 2024-04-28T12:38:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5739600640 10 client.4785 handle_command_reply: tid=0 2024-04-28T12:38:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5759bdb740 1 client.4785 shutdown 2024-04-28T12:38:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5759bdb740 1 -- 172.21.15.131:0/1724036833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff4dcb0 msgr2=0x55a43ff3f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5759bdb740 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff4dcb0 0x55a43ff3f150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57480565c0 tx=0x7f574808a8c0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5759bdb740 1 -- 172.21.15.131:0/1724036833 shutdown_connections 2024-04-28T12:38:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5759bdb740 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.157:6834/1937780695,v1:172.21.15.157:6835/1937780695] conn(0x55a43ff79920 0x55a43ff79d00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5759bdb740 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a43ff4dcb0 0x55a43ff3f150 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5759bdb740 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a43ff3c0a0 0x55a43ff4ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5759bdb740 1 --2- 172.21.15.131:0/1724036833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a43ff115e0 0x55a43ffba9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5739600640 0 client.4785 ms_handle_reset on v2:172.21.15.157:6834/1937780695 2024-04-28T12:38:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5759bdb740 1 -- 172.21.15.131:0/1724036833 >> 172.21.15.131:0/1724036833 conn(0x55a43fea5d10 msgr2=0x55a43ff52ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5759bdb740 1 -- 172.21.15.131:0/1724036833 shutdown_connections 2024-04-28T12:38:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.043+0000 7f5759bdb740 1 -- 172.21.15.131:0/1724036833 wait complete. 2024-04-28T12:38:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5759bdb740 20 client.4785 trim_cache size 0 max 16384 2024-04-28T12:38:04.043 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:38:04.043 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "fc0efa02-d382-4f6a-947e-cbe7ab2d58d1", 2024-04-28T12:38:04.043 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-04-28T12:38:04.044 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 34160, 2024-04-28T12:38:04.044 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-04-28T12:38:04.044 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-04-28T12:38:04.044 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-04-28T12:38:04.044 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 41.251273898999997, 2024-04-28T12:38:04.044 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 9, 2024-04-28T12:38:04.044 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 24, 2024-04-28T12:38:04.044 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 24, 2024-04-28T12:38:04.044 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 49.046396270999999 2024-04-28T12:38:04.044 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:38:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5756e00640 1 -- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f573006c8f0 msgr2=0x7f573006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5756e00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f573006c8f0 0x7f573006edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5740034000 tx=0x7f574002e000 comp rx=0 tx=0).stop 2024-04-28T12:38:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5756e00640 1 -- 172.21.15.131:0/199793845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57500b5370 msgr2=0x7f57500b7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5756e00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57500b5370 0x7f57500b7df0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5748007e60 tx=0x7f574800e4b0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5756e00640 1 -- 172.21.15.131:0/199793845 shutdown_connections 2024-04-28T12:38:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5756e00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f573006c8f0 0x7f573006edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5756e00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57500b5370 0x7f57500b7df0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5756e00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f575015eef0 0x7f575014e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5756e00640 1 --2- 172.21.15.131:0/199793845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57500d05b0 0x7f57500b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5756e00640 1 -- 172.21.15.131:0/199793845 >> 172.21.15.131:0/199793845 conn(0x7f575008e7e0 msgr2=0x7f57500cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:04.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5756e00640 1 -- 172.21.15.131:0/199793845 shutdown_connections 2024-04-28T12:38:04.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.044+0000 7f5756e00640 1 -- 172.21.15.131:0/199793845 wait complete. 2024-04-28T12:38:04.058 INFO:teuthology.run_tasks:Running task sequential... 2024-04-28T12:38:04.067 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-28T12:38:04.068 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-28T12:38:04.068 INFO:teuthology.task.print:Enabling mgr modules 2024-04-28T12:38:04.068 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-28T12:38:04.079 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-28T12:38:04.079 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-28T12:38:04.079 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-28T12:38:04.080 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-28T12:38:04.080 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-28T12:38:04.080 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-28T12:38:04.080 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-28T12:38:04.080 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-28T12:38:04.080 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-28T12:38:04.080 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-28T12:38:04.081 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-28T12:38:04.081 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-28T12:38:04.081 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-28T12:38:04.081 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-28T12:38:04.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:04.081 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-28T12:38:04.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:04.142 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-04-28T12:38:04.223 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-04-28T12:38:04.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:38:04.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:04.226 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-04-28T12:38:04.283 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-28T12:38:04.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:04.284 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:38:04.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:04.371 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-04-28T12:38:04.492 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-28T12:38:04.492 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T12:38:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.674+0000 7fb6c0600640 1 Processor -- start 2024-04-28T12:38:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.674+0000 7fb6c0600640 1 -- start start 2024-04-28T12:38:04.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.676+0000 7fb6c0600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6b813fc90 0x7fb6b8160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.676+0000 7fb6c0600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb6b80d0da0 0x7fb6b80d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.676+0000 7fb6c0600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6b815e3b0 con 0x7fb6b815df70 2024-04-28T12:38:04.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.676+0000 7fb6c0600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b80d1750 con 0x7fb6b80d0da0 2024-04-28T12:38:04.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.676+0000 7fb6c0600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b8161a10 con 0x7fb6b813fc90 2024-04-28T12:38:04.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.676+0000 7fb6bd200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb6b815df70 0x7fb6b8142c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44638/0 (socket says 172.21.15.131:44638) 2024-04-28T12:38:04.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.676+0000 7fb6bd200640 1 -- 172.21.15.131:0/1044984746 learned_addr learned my addr 172.21.15.131:0/1044984746 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:04.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6bdc00640 1 --2- 172.21.15.131:0/1044984746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6b813fc90 0x7fb6b8160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6be600640 1 --2- 172.21.15.131:0/1044984746 >> v2:172.21.15.157:3300/0 conn(0x7fb6b80d0da0 0x7fb6b80d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004364342 0 0) 0x7fb6b815e3b0 con 0x7fb6b815df70 2024-04-28T12:38:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6a0003520 con 0x7fb6b815df70 2024-04-28T12:38:04.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2554501186 0 0) 0x7fb6a0003520 con 0x7fb6b815df70 2024-04-28T12:38:04.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6b813fc90 msgr2=0x7fb6b8160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:04.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 --2- 172.21.15.131:0/1044984746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6b813fc90 0x7fb6b8160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 >> v2:172.21.15.157:3300/0 conn(0x7fb6b80d0da0 msgr2=0x7fb6b80d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:04.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 --2- 172.21.15.131:0/1044984746 >> v2:172.21.15.157:3300/0 conn(0x7fb6b80d0da0 0x7fb6b80d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b8161170 con 0x7fb6b815df70 2024-04-28T12:38:04.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fb6a4001670 con 0x7fb6b815df70 2024-04-28T12:38:04.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 >> v1:172.21.15.131:6789/0 conn(0x7fb6b815df70 legacy=0x7fb6b8142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:04.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 --2- 172.21.15.131:0/1044984746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6a00039b0 0x7fb6a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 --2- 172.21.15.131:0/1044984746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a0004800 0x7fb6a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 --2- 172.21.15.131:0/1044984746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6b813fc90 0x7fb6a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b80d1750 con 0x7fb6a00039b0 2024-04-28T12:38:04.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a000b9c0 con 0x7fb6a0004800 2024-04-28T12:38:04.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.677+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a000bb40 con 0x7fb6b813fc90 2024-04-28T12:38:04.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.678+0000 7fb6bdc00640 1 --2- 172.21.15.131:0/1044984746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6a00039b0 0x7fb6a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.678+0000 7fb6bd200640 1 --2- 172.21.15.131:0/1044984746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a0004800 0x7fb6a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.678+0000 7fb6bdc00640 1 -- 172.21.15.131:0/1044984746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6b813fc90 msgr2=0x7fb6a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:04.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.678+0000 7fb6bdc00640 1 --2- 172.21.15.131:0/1044984746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6b813fc90 0x7fb6a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.678+0000 7fb6bdc00640 1 -- 172.21.15.131:0/1044984746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a0004800 msgr2=0x7fb6a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:04.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.678+0000 7fb6bdc00640 1 --2- 172.21.15.131:0/1044984746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a0004800 0x7fb6a0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.678+0000 7fb6bdc00640 1 -- 172.21.15.131:0/1044984746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a0003520 con 0x7fb6a00039b0 2024-04-28T12:38:04.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.679+0000 7fb6bdc00640 1 --2- 172.21.15.131:0/1044984746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6a00039b0 0x7fb6a0003dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009ac0 tx=0x7fb6a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90199a82c1a1d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:04.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.679+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac036070 con 0x7fb6a00039b0 2024-04-28T12:38:04.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.679+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ac031400 con 0x7fb6a00039b0 2024-04-28T12:38:04.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.679+0000 7fb6b7e00640 1 -- 172.21.15.131:0/1044984746 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac009690 con 0x7fb6a00039b0 2024-04-28T12:38:04.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.680+0000 7fb6c0600640 1 -- 172.21.15.131:0/1044984746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6a00039b0 msgr2=0x7fb6a0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:04.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.680+0000 7fb6c0600640 1 --2- 172.21.15.131:0/1044984746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6a00039b0 0x7fb6a0003dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009ac0 tx=0x7fb6a0005830 comp rx=0 tx=0).stop 2024-04-28T12:38:04.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.680+0000 7fb6be600640 1 -- 172.21.15.131:0/1044984746 reap_dead start 2024-04-28T12:38:04.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.680+0000 7fb6c0600640 1 -- 172.21.15.131:0/1044984746 shutdown_connections 2024-04-28T12:38:04.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.680+0000 7fb6c0600640 1 -- 172.21.15.131:0/1044984746 >> 172.21.15.131:0/1044984746 conn(0x7fb6b80682a0 msgr2=0x7fb6b80686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:04.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.680+0000 7fb6c0600640 1 -- 172.21.15.131:0/1044984746 shutdown_connections 2024-04-28T12:38:04.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.680+0000 7fb6c0600640 1 -- 172.21.15.131:0/1044984746 wait complete. 2024-04-28T12:38:04.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.682+0000 7fb6c0600640 1 Processor -- start 2024-04-28T12:38:04.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.682+0000 7fb6c0600640 1 -- start start 2024-04-28T12:38:04.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.683+0000 7fb6c0600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a00039b0 0x7fb6b80b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.683+0000 7fb6c0600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6b8140070 0x7fb6b8158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.683+0000 7fb6c0600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b81404f0 0x7fb6b8159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.683+0000 7fb6c0600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b8142770 con 0x7fb6b81404f0 2024-04-28T12:38:04.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.683+0000 7fb6c0600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b8161730 con 0x7fb6a00039b0 2024-04-28T12:38:04.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.683+0000 7fb6c0600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b81433a0 con 0x7fb6b8140070 2024-04-28T12:38:04.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.683+0000 7fb6be600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b81404f0 0x7fb6b8159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.683+0000 7fb6be600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b81404f0 0x7fb6b8159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42994/0 (socket says 172.21.15.131:42994) 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.683+0000 7fb6be600640 1 -- 172.21.15.131:0/3296187028 learned_addr learned my addr 172.21.15.131:0/3296187028 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.684+0000 7fb6bdc00640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a00039b0 0x7fb6b80b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.684+0000 7fb6bd200640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6b8140070 0x7fb6b8158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.684+0000 7fb6be600640 1 -- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6b8140070 msgr2=0x7fb6b8158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.684+0000 7fb6be600640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6b8140070 0x7fb6b8158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.684+0000 7fb6be600640 1 -- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a00039b0 msgr2=0x7fb6b80b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.684+0000 7fb6be600640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a00039b0 0x7fb6b80b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.684+0000 7fb6be600640 1 -- 172.21.15.131:0/3296187028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b81409e0 con 0x7fb6b81404f0 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.684+0000 7fb6bdc00640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a00039b0 0x7fb6b80b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.684+0000 7fb6be600640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b81404f0 0x7fb6b8159330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8160a40 tx=0x7fb6b000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.684+0000 7fb6b5600640 1 -- 172.21.15.131:0/3296187028 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0018020 con 0x7fb6b81404f0 2024-04-28T12:38:04.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.685+0000 7fb6b5600640 1 -- 172.21.15.131:0/3296187028 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b000bce0 con 0x7fb6b81404f0 2024-04-28T12:38:04.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.685+0000 7fb6c0600640 1 -- 172.21.15.131:0/3296187028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b8140cc0 con 0x7fb6b81404f0 2024-04-28T12:38:04.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.685+0000 7fb6b5600640 1 -- 172.21.15.131:0/3296187028 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0014640 con 0x7fb6b81404f0 2024-04-28T12:38:04.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.685+0000 7fb6c0600640 1 -- 172.21.15.131:0/3296187028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b8141280 con 0x7fb6b81404f0 2024-04-28T12:38:04.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.686+0000 7fb6c0600640 1 -- 172.21.15.131:0/3296187028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b8069350 con 0x7fb6b81404f0 2024-04-28T12:38:04.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.687+0000 7fb6b5600640 1 -- 172.21.15.131:0/3296187028 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6b001b020 con 0x7fb6b81404f0 2024-04-28T12:38:04.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.688+0000 7fb6b5600640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb69806c8f0 0x7fb69806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:04.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.688+0000 7fb6b5600640 1 -- 172.21.15.131:0/3296187028 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb6b008c640 con 0x7fb6b81404f0 2024-04-28T12:38:04.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.688+0000 7fb6bdc00640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb69806c8f0 0x7fb69806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:04.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.689+0000 7fb6bdc00640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb69806c8f0 0x7fb69806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac033000 tx=0x7fb6ac008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:04.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.692+0000 7fb6b5600640 1 -- 172.21.15.131:0/3296187028 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb6b0022020 con 0x7fb6b81404f0 2024-04-28T12:38:04.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.936+0000 7fb6c0600640 1 -- 172.21.15.131:0/3296187028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb6b8073cb0 con 0x7fb6b81404f0 2024-04-28T12:38:04.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.937+0000 7fb6b5600640 1 -- 172.21.15.131:0/3296187028 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fb6b005a7e0 con 0x7fb6b81404f0 2024-04-28T12:38:04.933 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-28T12:38:04.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.939+0000 7fb6c0600640 1 -- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb69806c8f0 msgr2=0x7fb69806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:04.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.939+0000 7fb6c0600640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb69806c8f0 0x7fb69806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac033000 tx=0x7fb6ac008030 comp rx=0 tx=0).stop 2024-04-28T12:38:04.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.940+0000 7fb6c0600640 1 -- 172.21.15.131:0/3296187028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b81404f0 msgr2=0x7fb6b8159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:04.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.940+0000 7fb6c0600640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b81404f0 0x7fb6b8159330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8160a40 tx=0x7fb6b000b480 comp rx=0 tx=0).stop 2024-04-28T12:38:04.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.940+0000 7fb6c0600640 1 -- 172.21.15.131:0/3296187028 shutdown_connections 2024-04-28T12:38:04.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.940+0000 7fb6c0600640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb69806c8f0 0x7fb69806edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.940+0000 7fb6c0600640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b81404f0 0x7fb6b8159330 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.940+0000 7fb6c0600640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6b8140070 0x7fb6b8158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.940+0000 7fb6c0600640 1 --2- 172.21.15.131:0/3296187028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6a00039b0 0x7fb6b80b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:04.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.940+0000 7fb6c0600640 1 -- 172.21.15.131:0/3296187028 >> 172.21.15.131:0/3296187028 conn(0x7fb6b80682a0 msgr2=0x7fb6b80d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:04.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.940+0000 7fb6c0600640 1 -- 172.21.15.131:0/3296187028 shutdown_connections 2024-04-28T12:38:04.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:04.940+0000 7fb6c0600640 1 -- 172.21.15.131:0/3296187028 wait complete. 2024-04-28T12:38:04.956 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-28T12:38:04.956 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T12:38:04.956 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-04-28T12:38:04.956 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-28T12:38:04.956 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-28T12:38:04.956 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-28T12:38:04.956 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-28T12:38:04.956 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T12:38:04.956 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-28T12:38:04.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:04.957 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-04-28T12:38:05.006 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T12:38:05.007 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T12:38:05.007 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T12:38:05.007 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:05.007 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-04-28T12:38:05.007 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:05.007 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:38:05.007 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:05.007 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-04-28T12:38:05.007 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:38:05.007 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:05.007 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-04-28T12:38:05.007 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:38:05.008 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:05.008 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T12:38:05.008 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:05.008 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T12:38:05.008 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42490sec preferred_lft 42490sec 2024-04-28T12:38:05.008 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-04-28T12:38:05.008 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:05.009 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-28T12:38:05.009 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:05.009 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:38:05.009 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add name ceph-brx type bridge 2024-04-28T12:38:05.009 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr flush dev ceph-brx 2024-04-28T12:38:05.009 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set ceph-brx up 2024-04-28T12:38:05.010 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-28T12:38:05.010 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:38:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:05.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:05.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:05.251 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T12:38:05.318 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-04-28T12:38:05.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:05.322 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-04-28T12:38:05.371 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-04-28T12:38:05.371 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T12:38:05.372 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T12:38:05.372 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T12:38:05.372 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T12:38:05.373 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:05.373 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:38:05.373 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T12:38:05.373 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T12:38:05.373 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T12:38:05.373 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:38:05.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:05.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:05.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:05.580 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-04-28T12:38:05.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:05.629 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-04-28T12:38:05.679 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:05.679 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:38:05.679 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:38:05.679 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-28T12:38:05.679 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:38:05.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:05.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:05.862 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-28T12:38:05.862 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:05.862 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:38:05.862 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-28T12:38:05.862 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-28T12:38:05.862 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-28T12:38:05.862 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-28T12:38:05.862 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-28T12:38:05.862 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:38:05.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:06.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:06.123 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:06.124 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:38:06.124 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-04-28T12:38:06.124 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.0 master ceph-brx 2024-04-28T12:38:06.124 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:38:06.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:06.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:06.327 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-28T12:38:06.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:38:06.327 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-28T12:38:06.370 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-28T12:38:06.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:38:06.370 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-28T12:38:06.462 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-04-28T12:38:06.548 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:38:06.605 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-04-28T12:38:06.605 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-04-28T12:38:06.605 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-04-28T12:38:06.605 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-04-28T12:38:06.605 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-04-28T12:38:06.605 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-04-28T12:38:06.605 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-04-28T12:38:06.605 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-04-28T12:38:06.605 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-04-28T12:38:06.605 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-04-28T12:38:06.605 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/selinux 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-04-28T12:38:06.606 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-04-28T12:38:06.607 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-04-28T12:38:06.607 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-04-28T12:38:06.607 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-04-28T12:38:06.607 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-04-28T12:38:06.607 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-04-28T12:38:06.607 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:38:06.607 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:38:06.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:06.608 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-04-28T12:38:06.659 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-28T12:38:06.659 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-28T12:38:06.665 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-04-28T12:38:06.732 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:38:06.774 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-04-28T12:38:06.774 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-04-28T12:38:06.774 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-04-28T12:38:06.774 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/selinux 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-04-28T12:38:06.775 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:38:06.776 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:38:06.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:06.777 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-04-28T12:38:06.834 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-04-28T12:38:06.837+0000 7fe43fc3c480 -1 init, newargv = 0x55aa4754d540 newargc=15 2024-04-28T12:38:06.834 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[43076]: starting ceph client 2024-04-28T12:38:06.845 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[43076]: starting fuse 2024-04-28T12:38:06.861 INFO:teuthology.orchestra.run.smithi131.stdout:41 2024-04-28T12:38:06.861 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-28T12:38:06.861 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-04-28T12:38:06.862 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-04-28T12:38:06.862 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-04-28T12:38:06.862 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-04-28T12:38:06.862 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-04-28T12:38:06.862 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:38:06.862 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-04-28T12:38:06.862 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-28T12:38:06.862 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-04-28T12:38:06.863 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-28T12:38:06.863 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:38:06.863 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-04-28T12:38:06.863 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-04-28T12:38:06.863 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-04-28T12:38:06.863 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-04-28T12:38:06.863 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:38:06.864 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-04-28T12:38:06.864 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-28T12:38:06.864 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-28T12:38:06.864 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-28T12:38:06.864 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-04-28T12:38:06.864 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-04-28T12:38:06.864 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-04-28T12:38:06.865 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-28T12:38:06.865 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:38:06.865 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-04-28T12:38:06.865 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-04-28T12:38:07.054 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.43076.asok 2024-04-28T12:38:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:07.071 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43076.asok 2024-04-28T12:38:07.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:07.072 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43076.asok status 2024-04-28T12:38:07.298 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:38:07.298 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-04-28T12:38:07.298 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-28T12:38:07.298 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-28T12:38:07.298 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-04-28T12:38:07.298 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-04-28T12:38:07.298 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "43076", 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4797, 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 4797 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-04-28T12:38:07.299 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3919702858 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3919702858 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.4797 192.168.0.1:0/3919702858", 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/3919702858", 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 9, 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 24, 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-04-28T12:38:07.300 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-04-28T12:38:07.301 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-04-28T12:38:07.301 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:38:07.311 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T12:38:07.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.517+0000 7fb1b9c00640 1 Processor -- start 2024-04-28T12:38:07.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.517+0000 7fb1b9c00640 1 -- start start 2024-04-28T12:38:07.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.518+0000 7fb1b9c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1b40d17f0 0x7fb1b415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:07.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.518+0000 7fb1b9c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1b40d5840 0x7fb1b415f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:07.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.519+0000 7fb1b9c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb1b40b5f70 0x7fb1b4162ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:07.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.519+0000 7fb1b9c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b40d5cd0 con 0x7fb1b40d17f0 2024-04-28T12:38:07.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.519+0000 7fb1b9c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b40d3740 con 0x7fb1b40b5f70 2024-04-28T12:38:07.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.519+0000 7fb1b9c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b415f8d0 con 0x7fb1b40d5840 2024-04-28T12:38:07.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.519+0000 7fb1b3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1b40d17f0 0x7fb1b415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:07.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.519+0000 7fb1b3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1b40d17f0 0x7fb1b415ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43002/0 (socket says 172.21.15.131:43002) 2024-04-28T12:38:07.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.519+0000 7fb1b3e00640 1 -- 172.21.15.131:0/1759013754 learned_addr learned my addr 172.21.15.131:0/1759013754 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:07.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.519+0000 7fb1b3400640 1 --2- 172.21.15.131:0/1759013754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1b40d5840 0x7fb1b415f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:07.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.519+0000 7fb1b9200640 1 --2- 172.21.15.131:0/1759013754 >> v2:172.21.15.157:3300/0 conn(0x7fb1b40b5f70 0x7fb1b4162ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:07.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.520+0000 7fb1b3e00640 1 -- 172.21.15.131:0/1759013754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1b40d5840 msgr2=0x7fb1b415f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:07.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.520+0000 7fb1b3e00640 1 --2- 172.21.15.131:0/1759013754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1b40d5840 0x7fb1b415f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:07.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.520+0000 7fb1b3e00640 1 -- 172.21.15.131:0/1759013754 >> v2:172.21.15.157:3300/0 conn(0x7fb1b40b5f70 msgr2=0x7fb1b4162ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:07.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.520+0000 7fb1b3e00640 1 --2- 172.21.15.131:0/1759013754 >> v2:172.21.15.157:3300/0 conn(0x7fb1b40b5f70 0x7fb1b4162ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:07.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.520+0000 7fb1b3e00640 1 -- 172.21.15.131:0/1759013754 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b40d39e0 con 0x7fb1b40d17f0 2024-04-28T12:38:07.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.521+0000 7fb1b3e00640 1 --2- 172.21.15.131:0/1759013754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b40d17f0 0x7fb1b415ee50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb1b408eda0 tx=0x7fb1a402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c501cd86deb496f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:07.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.521+0000 7fb1b2a00640 1 -- 172.21.15.131:0/1759013754 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4037070 con 0x7fb1b40d17f0 2024-04-28T12:38:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.521+0000 7fb1b2a00640 1 -- 172.21.15.131:0/1759013754 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a4032440 con 0x7fb1b40d17f0 2024-04-28T12:38:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.521+0000 7fb1b2a00640 1 -- 172.21.15.131:0/1759013754 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4032810 con 0x7fb1b40d17f0 2024-04-28T12:38:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.522+0000 7fb1b9c00640 1 -- 172.21.15.131:0/1759013754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b40d17f0 msgr2=0x7fb1b415ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.522+0000 7fb1b9c00640 1 --2- 172.21.15.131:0/1759013754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b40d17f0 0x7fb1b415ee50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb1b408eda0 tx=0x7fb1a402a140 comp rx=0 tx=0).stop 2024-04-28T12:38:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.522+0000 7fb1b9c00640 1 -- 172.21.15.131:0/1759013754 shutdown_connections 2024-04-28T12:38:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.522+0000 7fb1b9c00640 1 --2- 172.21.15.131:0/1759013754 >> v2:172.21.15.157:3300/0 conn(0x7fb1b40b5f70 0x7fb1b4162ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.522+0000 7fb1b9c00640 1 --2- 172.21.15.131:0/1759013754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1b40d5840 0x7fb1b415f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.522+0000 7fb1b9c00640 1 --2- 172.21.15.131:0/1759013754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b40d17f0 0x7fb1b415ee50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.522+0000 7fb1b9c00640 1 -- 172.21.15.131:0/1759013754 >> 172.21.15.131:0/1759013754 conn(0x7fb1b40d4d20 msgr2=0x7fb1b40d5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.522+0000 7fb1b9c00640 1 -- 172.21.15.131:0/1759013754 shutdown_connections 2024-04-28T12:38:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.522+0000 7fb1b9c00640 1 -- 172.21.15.131:0/1759013754 wait complete. 2024-04-28T12:38:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.523+0000 7fb1b9c00640 1 Processor -- start 2024-04-28T12:38:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.524+0000 7fb1b9c00640 1 -- start start 2024-04-28T12:38:07.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.524+0000 7fb1b9c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1b40b5f70 0x7fb1b4162300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:07.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.524+0000 7fb1b9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b40d17f0 0x7fb1b4154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:07.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b9c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1b41549a0 0x7fb1b406cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:07.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b9c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b40d3e40 con 0x7fb1b40d17f0 2024-04-28T12:38:07.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b9c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b40d3cc0 con 0x7fb1b41549a0 2024-04-28T12:38:07.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b9c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b40b5990 con 0x7fb1b40b5f70 2024-04-28T12:38:07.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b40d17f0 0x7fb1b4154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:07.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b40d17f0 0x7fb1b4154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43008/0 (socket says 172.21.15.131:43008) 2024-04-28T12:38:07.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b3400640 1 -- 172.21.15.131:0/3920133210 learned_addr learned my addr 172.21.15.131:0/3920133210 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b3e00640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1b40b5f70 0x7fb1b4162300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b9200640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1b41549a0 0x7fb1b406cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b3400640 1 -- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1b40b5f70 msgr2=0x7fb1b4162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b3400640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1b40b5f70 0x7fb1b4162300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b3400640 1 -- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1b41549a0 msgr2=0x7fb1b406cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b3400640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1b41549a0 0x7fb1b406cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.525+0000 7fb1b3400640 1 -- 172.21.15.131:0/3920133210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b40cd410 con 0x7fb1b40d17f0 2024-04-28T12:38:07.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.526+0000 7fb1b3400640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b40d17f0 0x7fb1b4154460 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000d5f0 tx=0x7fb1a000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.526+0000 7fb1abe00640 1 -- 172.21.15.131:0/3920133210 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00125f0 con 0x7fb1b40d17f0 2024-04-28T12:38:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.526+0000 7fb1abe00640 1 -- 172.21.15.131:0/3920133210 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a0019070 con 0x7fb1b40d17f0 2024-04-28T12:38:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.526+0000 7fb1b9c00640 1 -- 172.21.15.131:0/3920133210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b40cd6b0 con 0x7fb1b40d17f0 2024-04-28T12:38:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.526+0000 7fb1abe00640 1 -- 172.21.15.131:0/3920133210 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00129d0 con 0x7fb1b40d17f0 2024-04-28T12:38:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.526+0000 7fb1b9c00640 1 -- 172.21.15.131:0/3920133210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b40cdc70 con 0x7fb1b40d17f0 2024-04-28T12:38:07.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.527+0000 7fb1b9c00640 1 -- 172.21.15.131:0/3920133210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b4069350 con 0x7fb1b40d17f0 2024-04-28T12:38:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.528+0000 7fb1abe00640 1 -- 172.21.15.131:0/3920133210 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1a0012b70 con 0x7fb1b40d17f0 2024-04-28T12:38:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.529+0000 7fb1abe00640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb19406c8f0 0x7fb19406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.529+0000 7fb1abe00640 1 -- 172.21.15.131:0/3920133210 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7fb1a008c440 con 0x7fb1b40d17f0 2024-04-28T12:38:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.530+0000 7fb1b3e00640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb19406c8f0 0x7fb19406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:07.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.531+0000 7fb1b3e00640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb19406c8f0 0x7fb19406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4034000 tx=0x7fb1a402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:07.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.534+0000 7fb1abe00640 1 -- 172.21.15.131:0/3920133210 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb1b4069350 con 0x7fb1b40d17f0 2024-04-28T12:38:07.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.777+0000 7fb1b9c00640 1 -- 172.21.15.131:0/3920133210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb1b4069a60 con 0x7fb1b40d17f0 2024-04-28T12:38:07.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.778+0000 7fb1abe00640 1 -- 172.21.15.131:0/3920133210 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fb1a005a3d0 con 0x7fb1b40d17f0 2024-04-28T12:38:07.775 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-28T12:38:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.781+0000 7fb1b9c00640 1 -- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb19406c8f0 msgr2=0x7fb19406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.781+0000 7fb1b9c00640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb19406c8f0 0x7fb19406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4034000 tx=0x7fb1a402e000 comp rx=0 tx=0).stop 2024-04-28T12:38:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.781+0000 7fb1b9c00640 1 -- 172.21.15.131:0/3920133210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b40d17f0 msgr2=0x7fb1b4154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.781+0000 7fb1b9c00640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b40d17f0 0x7fb1b4154460 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000d5f0 tx=0x7fb1a000dab0 comp rx=0 tx=0).stop 2024-04-28T12:38:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.781+0000 7fb1b9c00640 1 -- 172.21.15.131:0/3920133210 shutdown_connections 2024-04-28T12:38:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.781+0000 7fb1b9c00640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb19406c8f0 0x7fb19406edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.781+0000 7fb1b9c00640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1b41549a0 0x7fb1b406cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.781+0000 7fb1b9c00640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b40d17f0 0x7fb1b4154460 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.781+0000 7fb1b9c00640 1 --2- 172.21.15.131:0/3920133210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb1b40b5f70 0x7fb1b4162300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.781+0000 7fb1b9c00640 1 -- 172.21.15.131:0/3920133210 >> 172.21.15.131:0/3920133210 conn(0x7fb1b40d4d20 msgr2=0x7fb1b406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:07.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.781+0000 7fb1b9c00640 1 -- 172.21.15.131:0/3920133210 shutdown_connections 2024-04-28T12:38:07.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:07.781+0000 7fb1b9c00640 1 -- 172.21.15.131:0/3920133210 wait complete. 2024-04-28T12:38:07.797 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-28T12:38:07.797 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T12:38:07.797 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-04-28T12:38:07.797 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-28T12:38:07.797 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-28T12:38:07.797 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-28T12:38:07.797 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-28T12:38:07.797 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T12:38:07.798 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-28T12:38:07.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:07.798 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-04-28T12:38:07.846 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:07.847 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42487sec preferred_lft 42487sec 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 7e:da:99:10:fa:2a brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::682c:c6ff:fe45:6ca5/64 scope link 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 7e:da:99:10:fa:2a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::7cda:99ff:fe10:fa2a/64 scope link 2024-04-28T12:38:07.848 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:07.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:07.849 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T12:38:07.916 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-04-28T12:38:07.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:07.920 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-04-28T12:38:07.964 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-04-28T12:38:07.964 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T12:38:07.964 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T12:38:07.965 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T12:38:07.965 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T12:38:07.965 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:07.965 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:38:07.965 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T12:38:07.965 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T12:38:07.965 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T12:38:07.965 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:38:08.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:08.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:08.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:08.217 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-04-28T12:38:08.255 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-28T12:38:08.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:08.256 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-04-28T12:38:08.345 INFO:teuthology.orchestra.run.smithi131.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-28T12:38:08.346 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:08.346 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:38:08.346 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T12:38:08.346 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-28T12:38:08.346 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:38:08.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:08.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:08.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:08.571 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-28T12:38:08.647 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T12:38:08.647 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T12:38:08.647 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T12:38:08.647 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:08.647 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-04-28T12:38:08.647 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:08.647 INFO:teuthology.orchestra.run.smithi131.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T12:38:08.647 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 1e:be:3a:29:40:54 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-28T12:38:08.647 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-28T12:38:08.648 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:08.648 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::1cbe:3aff:fe29:4054/64 scope link 2024-04-28T12:38:08.648 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:08.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:08.651 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-28T12:38:08.734 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T12:38:08.734 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T12:38:08.735 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T12:38:08.735 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:08.735 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-04-28T12:38:08.735 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:08.735 INFO:teuthology.orchestra.run.smithi131.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T12:38:08.735 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 1e:be:3a:29:40:54 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-28T12:38:08.736 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-28T12:38:08.736 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:08.736 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::1cbe:3aff:fe29:4054/64 scope link 2024-04-28T12:38:08.736 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:08.738 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-28T12:38:08.739 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:08.739 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:38:08.739 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-28T12:38:08.739 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-28T12:38:08.740 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-28T12:38:08.740 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-28T12:38:08.740 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-28T12:38:08.740 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:38:08.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:09.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:09.020 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:09.020 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:38:09.020 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.1 up 2024-04-28T12:38:09.020 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.1 master ceph-brx 2024-04-28T12:38:09.020 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:38:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:09.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:09.228 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-28T12:38:09.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:38:09.228 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-28T12:38:09.271 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-28T12:38:09.271 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:38:09.271 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-28T12:38:09.364 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-04-28T12:38:09.455 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:38:09.509 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-04-28T12:38:09.509 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-04-28T12:38:09.509 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-04-28T12:38:09.509 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-04-28T12:38:09.509 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-04-28T12:38:09.509 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-04-28T12:38:09.509 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/selinux 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-04-28T12:38:09.510 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T12:38:09.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T12:38:09.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:09.513 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-04-28T12:38:09.564 INFO:teuthology.orchestra.run.smithi131.stdout:41 2024-04-28T12:38:09.564 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-04-28T12:38:09.565 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-28T12:38:09.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-04-28T12:38:09.645 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:38:09.686 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-04-28T12:38:09.686 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-04-28T12:38:09.686 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-04-28T12:38:09.686 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/selinux 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-04-28T12:38:09.687 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:38:09.688 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-28T12:38:09.689 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T12:38:09.689 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T12:38:09.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:09.690 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-04-28T12:38:09.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi131.stderr:2024-04-28T12:38:09.747+0000 7f355531c480 -1 init, newargv = 0x55df05cb7410 newargc=15 2024-04-28T12:38:09.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi131.stderr:ceph-fuse[43574]: starting ceph client 2024-04-28T12:38:09.754 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi131.stderr:ceph-fuse[43574]: starting fuse 2024-04-28T12:38:09.772 INFO:teuthology.orchestra.run.smithi131.stdout:41 2024-04-28T12:38:09.773 INFO:teuthology.orchestra.run.smithi131.stdout:43 2024-04-28T12:38:09.773 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 43] 2024-04-28T12:38:09.773 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-04-28T12:38:09.773 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-04-28T12:38:09.773 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-04-28T12:38:09.773 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-04-28T12:38:09.773 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-04-28T12:38:09.773 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:38:09.773 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-04-28T12:38:09.773 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-28T12:38:09.773 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-04-28T12:38:09.774 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-04-28T12:38:09.775 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-04-28T12:38:09.775 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-28T12:38:09.775 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:38:09.775 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.2")) 2024-04-28T12:38:09.775 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-04-28T12:38:09.961 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.2.43574.asok 2024-04-28T12:38:09.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:09.978 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.43574.asok 2024-04-28T12:38:09.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:09.978 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.43574.asok status 2024-04-28T12:38:10.203 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:38:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-04-28T12:38:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-28T12:38:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-28T12:38:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "2", 2024-04-28T12:38:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-04-28T12:38:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-28T12:38:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "43574", 2024-04-28T12:38:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-04-28T12:38:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:38:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-04-28T12:38:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-04-28T12:38:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4806, 2024-04-28T12:38:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 4806 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.2:0", 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 643984057 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.2:0", 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 643984057 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:38:10.205 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.4806 192.168.0.2:0/643984057", 2024-04-28T12:38:10.206 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.2:0/643984057", 2024-04-28T12:38:10.206 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-04-28T12:38:10.206 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 9, 2024-04-28T12:38:10.206 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 26, 2024-04-28T12:38:10.206 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-04-28T12:38:10.206 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-04-28T12:38:10.206 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-04-28T12:38:10.206 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:38:10.219 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T12:38:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.421+0000 7f429a600640 1 Processor -- start 2024-04-28T12:38:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.422+0000 7f429a600640 1 -- start start 2024-04-28T12:38:10.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.423+0000 7f429a600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940708f0 0x7f4294070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:10.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.423+0000 7f429a600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4294071210 con 0x7f42940cffa0 2024-04-28T12:38:10.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.423+0000 7f429a600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42940d1860 con 0x7f429415e800 2024-04-28T12:38:10.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.423+0000 7f429a600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f429406d460 con 0x7f42940708f0 2024-04-28T12:38:10.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.423+0000 7f4293e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f42940cffa0 0x7f42940d0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44648/0 (socket says 172.21.15.131:44648) 2024-04-28T12:38:10.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.423+0000 7f4293e00640 1 -- 172.21.15.131:0/212208498 learned_addr learned my addr 172.21.15.131:0/212208498 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:10.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.424+0000 7f4299200640 1 --2- 172.21.15.131:0/212208498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940708f0 0x7f4294070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:10.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.424+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037690113 0 0) 0x7f4294071210 con 0x7f42940cffa0 2024-04-28T12:38:10.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.424+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4278003520 con 0x7f42940cffa0 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.424+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 634533825 0 0) 0x7f4278003520 con 0x7f42940cffa0 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.424+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940708f0 msgr2=0x7f4294070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.424+0000 7f4293400640 1 --2- 172.21.15.131:0/212208498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940708f0 0x7f4294070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.424+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 >> v1:172.21.15.157:6789/0 conn(0x7f429415e800 legacy=0x7f429415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.424+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42940d1b40 con 0x7f42940cffa0 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.424+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f427c001670 con 0x7f42940cffa0 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.424+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 >> v1:172.21.15.131:6789/0 conn(0x7f42940cffa0 legacy=0x7f42940d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4293400640 1 --2- 172.21.15.131:0/212208498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780039e0 0x7f4278003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4293400640 1 --2- 172.21.15.131:0/212208498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940708f0 0x7f4278004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4293400640 1 --2- 172.21.15.131:0/212208498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4278004d40 0x7f42780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f429406d460 con 0x7f4278004d40 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f427800baa0 con 0x7f42780039e0 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f427800bca0 con 0x7f42940708f0 2024-04-28T12:38:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4299c00640 1 --2- 172.21.15.131:0/212208498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4278004d40 0x7f42780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4293e00640 1 --2- 172.21.15.131:0/212208498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940708f0 0x7f4278004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4299200640 1 --2- 172.21.15.131:0/212208498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780039e0 0x7f4278003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4299c00640 1 -- 172.21.15.131:0/212208498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940708f0 msgr2=0x7f4278004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4299c00640 1 --2- 172.21.15.131:0/212208498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940708f0 0x7f4278004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4299c00640 1 -- 172.21.15.131:0/212208498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780039e0 msgr2=0x7f4278003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4299c00640 1 --2- 172.21.15.131:0/212208498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42780039e0 0x7f4278003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.425+0000 7f4299c00640 1 -- 172.21.15.131:0/212208498 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4278003520 con 0x7f4278004d40 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.426+0000 7f4299c00640 1 --2- 172.21.15.131:0/212208498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4278004d40 0x7f42780051c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f428c001720 tx=0x7f4278005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853b8363f772bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.427+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c001370 con 0x7f4278004d40 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.427+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f428c0052d0 con 0x7f4278004d40 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.427+0000 7f4293400640 1 -- 172.21.15.131:0/212208498 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c02abc0 con 0x7f4278004d40 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.427+0000 7f429a600640 1 -- 172.21.15.131:0/212208498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4278004d40 msgr2=0x7f42780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.427+0000 7f429a600640 1 --2- 172.21.15.131:0/212208498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4278004d40 0x7f42780051c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f428c001720 tx=0x7f4278005830 comp rx=0 tx=0).stop 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.427+0000 7f4299c00640 1 -- 172.21.15.131:0/212208498 reap_dead start 2024-04-28T12:38:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.427+0000 7f429a600640 1 -- 172.21.15.131:0/212208498 shutdown_connections 2024-04-28T12:38:10.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.427+0000 7f429a600640 1 -- 172.21.15.131:0/212208498 >> 172.21.15.131:0/212208498 conn(0x7f42940b46e0 msgr2=0x7f42940b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:10.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.427+0000 7f429a600640 1 -- 172.21.15.131:0/212208498 shutdown_connections 2024-04-28T12:38:10.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.427+0000 7f429a600640 1 -- 172.21.15.131:0/212208498 wait complete. 2024-04-28T12:38:10.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.429+0000 7f429a600640 1 Processor -- start 2024-04-28T12:38:10.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.429+0000 7f429a600640 1 -- start start 2024-04-28T12:38:10.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.430+0000 7f429a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4278004d40 0x7f42941eaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:10.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.430+0000 7f429a600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42940c66c0 0x7f42940c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:10.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.430+0000 7f429a600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940c83e0 0x7f42940c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:10.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.430+0000 7f429a600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f42940d1570 con 0x7f42940c66c0 2024-04-28T12:38:10.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.430+0000 7f429a600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f42940d2100 con 0x7f4278004d40 2024-04-28T12:38:10.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.430+0000 7f429a600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f42940b2360 con 0x7f42940c83e0 2024-04-28T12:38:10.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.430+0000 7f4293e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42940c66c0 0x7f42940c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:10.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.431+0000 7f4293e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42940c66c0 0x7f42940c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43032/0 (socket says 172.21.15.131:43032) 2024-04-28T12:38:10.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.431+0000 7f4293e00640 1 -- 172.21.15.131:0/2131599614 learned_addr learned my addr 172.21.15.131:0/2131599614 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.431+0000 7f4299200640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4278004d40 0x7f42941eaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.431+0000 7f4299c00640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940c83e0 0x7f42940c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.431+0000 7f4293e00640 1 -- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940c83e0 msgr2=0x7f42940c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.431+0000 7f4293e00640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940c83e0 0x7f42940c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.431+0000 7f4293e00640 1 -- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4278004d40 msgr2=0x7f42941eaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.431+0000 7f4293e00640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4278004d40 0x7f42941eaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.431+0000 7f4293e00640 1 -- 172.21.15.131:0/2131599614 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42940b5bf0 con 0x7f42940c66c0 2024-04-28T12:38:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.431+0000 7f4299200640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4278004d40 0x7f42941eaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.431+0000 7f4293e00640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42940c66c0 0x7f42940c6b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f427c00d810 tx=0x7f427c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.432+0000 7f4290c00640 1 -- 172.21.15.131:0/2131599614 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c019070 con 0x7f42940c66c0 2024-04-28T12:38:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.432+0000 7f4290c00640 1 -- 172.21.15.131:0/2131599614 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f427c0074e0 con 0x7f42940c66c0 2024-04-28T12:38:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.432+0000 7f4290c00640 1 -- 172.21.15.131:0/2131599614 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c004a00 con 0x7f42940c66c0 2024-04-28T12:38:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.432+0000 7f429a600640 1 -- 172.21.15.131:0/2131599614 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42940b5ed0 con 0x7f42940c66c0 2024-04-28T12:38:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.432+0000 7f429a600640 1 -- 172.21.15.131:0/2131599614 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42940b6490 con 0x7f42940c66c0 2024-04-28T12:38:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.433+0000 7f429a600640 1 -- 172.21.15.131:0/2131599614 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42940b8870 con 0x7f42940c66c0 2024-04-28T12:38:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.434+0000 7f4290c00640 1 -- 172.21.15.131:0/2131599614 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f427c01c070 con 0x7f42940c66c0 2024-04-28T12:38:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.435+0000 7f4290c00640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f427006c970 0x7f427006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.435+0000 7f4290c00640 1 -- 172.21.15.131:0/2131599614 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f427c08b880 con 0x7f42940c66c0 2024-04-28T12:38:10.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.435+0000 7f4299200640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f427006c970 0x7f427006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:10.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.436+0000 7f4299200640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f427006c970 0x7f427006ee30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4284007460 tx=0x7f428400d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:10.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.439+0000 7f4290c00640 1 -- 172.21.15.131:0/2131599614 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f42940b8870 con 0x7f42940c66c0 2024-04-28T12:38:10.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.688+0000 7f429a600640 1 -- 172.21.15.131:0/2131599614 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4294069a60 con 0x7f42940c66c0 2024-04-28T12:38:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.689+0000 7f4290c00640 1 -- 172.21.15.131:0/2131599614 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f427c0592d0 con 0x7f42940c66c0 2024-04-28T12:38:10.686 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-28T12:38:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.692+0000 7f429a600640 1 -- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f427006c970 msgr2=0x7f427006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.692+0000 7f429a600640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f427006c970 0x7f427006ee30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4284007460 tx=0x7f428400d040 comp rx=0 tx=0).stop 2024-04-28T12:38:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.692+0000 7f429a600640 1 -- 172.21.15.131:0/2131599614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42940c66c0 msgr2=0x7f42940c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.692+0000 7f429a600640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42940c66c0 0x7f42940c6b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f427c00d810 tx=0x7f427c00dcd0 comp rx=0 tx=0).stop 2024-04-28T12:38:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.692+0000 7f429a600640 1 -- 172.21.15.131:0/2131599614 shutdown_connections 2024-04-28T12:38:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.692+0000 7f429a600640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f427006c970 0x7f427006ee30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.692+0000 7f429a600640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f42940c83e0 0x7f42940c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.692+0000 7f429a600640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42940c66c0 0x7f42940c6b40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.692+0000 7f429a600640 1 --2- 172.21.15.131:0/2131599614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4278004d40 0x7f42941eaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.692+0000 7f429a600640 1 -- 172.21.15.131:0/2131599614 >> 172.21.15.131:0/2131599614 conn(0x7f42940b46e0 msgr2=0x7f42940b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.692+0000 7f429a600640 1 -- 172.21.15.131:0/2131599614 shutdown_connections 2024-04-28T12:38:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:10.692+0000 7f429a600640 1 -- 172.21.15.131:0/2131599614 wait complete. 2024-04-28T12:38:10.708 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-28T12:38:10.709 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T12:38:10.709 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-28T12:38:10.709 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-28T12:38:10.709 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-28T12:38:10.709 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-28T12:38:10.709 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-28T12:38:10.709 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T12:38:10.709 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-28T12:38:10.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:10.709 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-28T12:38:10.766 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T12:38:10.766 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T12:38:10.766 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T12:38:10.766 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:10.767 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T12:38:10.768 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42506sec preferred_lft 42506sec 2024-04-28T12:38:10.768 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-28T12:38:10.768 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:10.768 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-28T12:38:10.768 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:10.768 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-28T12:38:10.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2024-04-28T12:38:10.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2024-04-28T12:38:10.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2024-04-28T12:38:10.768 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-28T12:38:10.769 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-28T12:38:10.876 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:11.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:11.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:11.248 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T12:38:11.313 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-28T12:38:11.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:11.316 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-28T12:38:11.371 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-28T12:38:11.371 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T12:38:11.371 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T12:38:11.371 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T12:38:11.371 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T12:38:11.372 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:11.372 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-28T12:38:11.372 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T12:38:11.372 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T12:38:11.372 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T12:38:11.372 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-28T12:38:11.470 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:12.155 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:12.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:12.165 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-28T12:38:12.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:12.222 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-04-28T12:38:12.275 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:12.276 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-28T12:38:12.276 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T12:38:12.276 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-28T12:38:12.276 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-28T12:38:12.378 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:12.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:12.457 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-28T12:38:12.458 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:12.458 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-28T12:38:12.458 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-28T12:38:12.458 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-28T12:38:12.458 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-28T12:38:12.458 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-28T12:38:12.458 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-28T12:38:12.458 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-28T12:38:12.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:12.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:12.742 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:12.743 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-28T12:38:12.743 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-28T12:38:12.743 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2024-04-28T12:38:12.743 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-28T12:38:12.841 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:12.940 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:12.949 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-28T12:38:12.949 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:38:12.949 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-28T12:38:12.993 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-28T12:38:12.994 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:38:12.994 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-28T12:38:13.084 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-28T12:38:13.157 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:38:13.204 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-28T12:38:13.204 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-28T12:38:13.204 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-28T12:38:13.205 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-28T12:38:13.206 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-28T12:38:13.207 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T12:38:13.207 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T12:38:13.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:13.208 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-28T12:38:13.301 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-28T12:38:13.302 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-28T12:38:13.306 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-28T12:38:13.371 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:38:13.413 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-28T12:38:13.413 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-28T12:38:13.413 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-28T12:38:13.413 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-28T12:38:13.413 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-28T12:38:13.414 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-28T12:38:13.414 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-28T12:38:13.414 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-28T12:38:13.414 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-28T12:38:13.414 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-28T12:38:13.414 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-28T12:38:13.414 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-04-28T12:38:13.414 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-28T12:38:13.415 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-28T12:38:13.416 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-28T12:38:13.416 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-28T12:38:13.416 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-28T12:38:13.416 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-28T12:38:13.416 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T12:38:13.416 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T12:38:13.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:13.417 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-28T12:38:13.486 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:2024-04-28T12:38:13.489+0000 7f413e317480 -1 init, newargv = 0x55ae2b2fe270 newargc=15 2024-04-28T12:38:13.486 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[38251]: starting ceph client 2024-04-28T12:38:13.498 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi157.stderr:ceph-fuse[38251]: starting fuse 2024-04-28T12:38:13.505 INFO:teuthology.orchestra.run.smithi157.stdout:40 2024-04-28T12:38:13.505 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-04-28T12:38:13.505 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-28T12:38:13.505 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-28T12:38:13.505 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-28T12:38:13.505 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-28T12:38:13.505 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-28T12:38:13.506 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-28T12:38:13.507 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-28T12:38:13.507 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-28T12:38:13.507 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-28T12:38:13.507 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-28T12:38:13.507 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-28T12:38:13.507 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.1")) 2024-04-28T12:38:13.507 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-28T12:38:13.699 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.1.38251.asok 2024-04-28T12:38:13.704 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:13.716 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38251.asok 2024-04-28T12:38:13.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:13.716 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38251.asok status 2024-04-28T12:38:13.945 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-28T12:38:13.945 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-28T12:38:13.945 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-28T12:38:13.945 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-28T12:38:13.945 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "1", 2024-04-28T12:38:13.945 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-28T12:38:13.945 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "38251", 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 4597, 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 4597 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-28T12:38:13.946 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1906574757 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.1:0", 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 1906574757 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.4597 192.168.0.1:0/1906574757", 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.1:0/1906574757", 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 9, 2024-04-28T12:38:13.947 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 26, 2024-04-28T12:38:13.948 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-28T12:38:13.948 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-28T12:38:13.948 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-28T12:38:13.948 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-28T12:38:13.959 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T12:38:14.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.173+0000 7fba52c00640 1 Processor -- start 2024-04-28T12:38:14.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.173+0000 7fba52c00640 1 -- start start 2024-04-28T12:38:14.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.174+0000 7fba52c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c1463e0 0x7fba4c0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:14.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.174+0000 7fba52c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fba4c161460 0x7fba4c160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:14.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.174+0000 7fba52c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba4c0b50e0 con 0x7fba4c160d90 2024-04-28T12:38:14.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.174+0000 7fba52c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fba4c1611e0 con 0x7fba4c161460 2024-04-28T12:38:14.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.174+0000 7fba52c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba4c0b4560 con 0x7fba4c1463e0 2024-04-28T12:38:14.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.175+0000 7fba51800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c1463e0 0x7fba4c0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:14.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.175+0000 7fba50e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fba4c161460 0x7fba4c160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:14.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.175+0000 7fba51800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c1463e0 0x7fba4c0b58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:57904/0 (socket says 172.21.15.131:57904) 2024-04-28T12:38:14.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.175+0000 7fba51800640 1 -- 172.21.15.131:0/2599629097 learned_addr learned my addr 172.21.15.131:0/2599629097 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:14.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.175+0000 7fba52200640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fba4c160d90 0x7fba4c0cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.131:37480/0 (socket says 172.21.15.131:37480) 2024-04-28T12:38:14.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.176+0000 7fba51800640 1 -- 172.21.15.131:0/2599629097 >> v2:172.21.15.157:3300/0 conn(0x7fba4c161460 msgr2=0x7fba4c160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:14.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.176+0000 7fba51800640 1 --2- 172.21.15.131:0/2599629097 >> v2:172.21.15.157:3300/0 conn(0x7fba4c161460 0x7fba4c160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:14.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.176+0000 7fba51800640 1 -- 172.21.15.131:0/2599629097 >> v1:172.21.15.157:6789/0 conn(0x7fba4c160d90 legacy=0x7fba4c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:14.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.176+0000 7fba51800640 1 -- 172.21.15.131:0/2599629097 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c0b4840 con 0x7fba4c1463e0 2024-04-28T12:38:14.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.176+0000 7fba50e00640 1 --2- 172.21.15.131:0/2599629097 >> v2:172.21.15.157:3300/0 conn(0x7fba4c161460 0x7fba4c160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:38:14.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.176+0000 7fba4be00640 1 -- 172.21.15.131:0/2599629097 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 730216780 0 0) 0x7fba4c0b50e0 con 0x7fba4c160d90 2024-04-28T12:38:14.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.177+0000 7fba51800640 1 --2- 172.21.15.131:0/2599629097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c1463e0 0x7fba4c0b58b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fba40009870 tx=0x7fba4002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28b8a72642e9700d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:14.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.178+0000 7fba4be00640 1 -- 172.21.15.131:0/2599629097 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40037070 con 0x7fba4c1463e0 2024-04-28T12:38:14.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.178+0000 7fba4be00640 1 -- 172.21.15.131:0/2599629097 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba40032440 con 0x7fba4c1463e0 2024-04-28T12:38:14.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.178+0000 7fba4be00640 1 -- 172.21.15.131:0/2599629097 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40032810 con 0x7fba4c1463e0 2024-04-28T12:38:14.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.178+0000 7fba52c00640 1 -- 172.21.15.131:0/2599629097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c1463e0 msgr2=0x7fba4c0b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:14.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.178+0000 7fba52c00640 1 --2- 172.21.15.131:0/2599629097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c1463e0 0x7fba4c0b58b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fba40009870 tx=0x7fba4002a1c0 comp rx=0 tx=0).stop 2024-04-28T12:38:14.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.179+0000 7fba52c00640 1 -- 172.21.15.131:0/2599629097 shutdown_connections 2024-04-28T12:38:14.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.179+0000 7fba52c00640 1 --2- 172.21.15.131:0/2599629097 >> v2:172.21.15.157:3300/0 conn(0x7fba4c161460 0x7fba4c160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:14.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.179+0000 7fba52c00640 1 --2- 172.21.15.131:0/2599629097 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c1463e0 0x7fba4c0b58b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:14.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.179+0000 7fba52c00640 1 -- 172.21.15.131:0/2599629097 >> 172.21.15.131:0/2599629097 conn(0x7fba4c0b26b0 msgr2=0x7fba4c0b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:14.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.179+0000 7fba52c00640 1 -- 172.21.15.131:0/2599629097 shutdown_connections 2024-04-28T12:38:14.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.179+0000 7fba52c00640 1 -- 172.21.15.131:0/2599629097 wait complete. 2024-04-28T12:38:14.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.180+0000 7fba52c00640 1 Processor -- start 2024-04-28T12:38:14.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.181+0000 7fba52c00640 1 -- start start 2024-04-28T12:38:14.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.181+0000 7fba52c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c1463e0 0x7fba4c145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:14.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba52c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba4c160d90 0x7fba4c1583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:14.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba52c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c161460 0x7fba4c1426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:14.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba52c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c1467c0 con 0x7fba4c160d90 2024-04-28T12:38:14.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba52c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c0b4e00 con 0x7fba4c1463e0 2024-04-28T12:38:14.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba52c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba4c0b5df0 con 0x7fba4c161460 2024-04-28T12:38:14.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba50e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba4c160d90 0x7fba4c1583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:14.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba50e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba4c160d90 0x7fba4c1583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54774/0 (socket says 172.21.15.131:54774) 2024-04-28T12:38:14.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba50e00640 1 -- 172.21.15.131:0/2393606196 learned_addr learned my addr 172.21.15.131:0/2393606196 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:14.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba51800640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c1463e0 0x7fba4c145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:14.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba52200640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c161460 0x7fba4c1426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:14.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba50e00640 1 -- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c161460 msgr2=0x7fba4c1426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:14.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba50e00640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c161460 0x7fba4c1426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba50e00640 1 -- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c1463e0 msgr2=0x7fba4c145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba50e00640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c1463e0 0x7fba4c145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba50e00640 1 -- 172.21.15.131:0/2393606196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c141650 con 0x7fba4c160d90 2024-04-28T12:38:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.182+0000 7fba52200640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c161460 0x7fba4c1426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.183+0000 7fba50e00640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba4c160d90 0x7fba4c1583d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fba3800ec10 tx=0x7fba3800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.183+0000 7fba49600640 1 -- 172.21.15.131:0/2393606196 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba38017070 con 0x7fba4c160d90 2024-04-28T12:38:14.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.183+0000 7fba49600640 1 -- 172.21.15.131:0/2393606196 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba3800cc30 con 0x7fba4c160d90 2024-04-28T12:38:14.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.183+0000 7fba52c00640 1 -- 172.21.15.131:0/2393606196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4c141930 con 0x7fba4c160d90 2024-04-28T12:38:14.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.183+0000 7fba49600640 1 -- 172.21.15.131:0/2393606196 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba38013470 con 0x7fba4c160d90 2024-04-28T12:38:14.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.183+0000 7fba52c00640 1 -- 172.21.15.131:0/2393606196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba4c141ef0 con 0x7fba4c160d90 2024-04-28T12:38:14.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.184+0000 7fba52c00640 1 -- 172.21.15.131:0/2393606196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba4c069350 con 0x7fba4c160d90 2024-04-28T12:38:14.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.185+0000 7fba49600640 1 -- 172.21.15.131:0/2393606196 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba3801a070 con 0x7fba4c160d90 2024-04-28T12:38:14.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.186+0000 7fba49600640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fba2c06c8f0 0x7fba2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:14.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.186+0000 7fba49600640 1 -- 172.21.15.131:0/2393606196 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fba3808c210 con 0x7fba4c160d90 2024-04-28T12:38:14.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.187+0000 7fba51800640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fba2c06c8f0 0x7fba2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:14.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.188+0000 7fba51800640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fba2c06c8f0 0x7fba2c06edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fba40034000 tx=0x7fba4002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:14.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.191+0000 7fba49600640 1 -- 172.21.15.131:0/2393606196 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fba3805a230 con 0x7fba4c160d90 2024-04-28T12:38:14.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.424+0000 7fba52c00640 1 -- 172.21.15.131:0/2393606196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fba4c0b4050 con 0x7fba4c160d90 2024-04-28T12:38:14.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.425+0000 7fba49600640 1 -- 172.21.15.131:0/2393606196 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fba38059b30 con 0x7fba4c160d90 2024-04-28T12:38:14.421 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-28T12:38:14.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.427+0000 7fba52c00640 1 -- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fba2c06c8f0 msgr2=0x7fba2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:14.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.427+0000 7fba52c00640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fba2c06c8f0 0x7fba2c06edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fba40034000 tx=0x7fba4002e000 comp rx=0 tx=0).stop 2024-04-28T12:38:14.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.427+0000 7fba52c00640 1 -- 172.21.15.131:0/2393606196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba4c160d90 msgr2=0x7fba4c1583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:14.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.427+0000 7fba52c00640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba4c160d90 0x7fba4c1583d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fba3800ec10 tx=0x7fba3800c520 comp rx=0 tx=0).stop 2024-04-28T12:38:14.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.427+0000 7fba52c00640 1 -- 172.21.15.131:0/2393606196 shutdown_connections 2024-04-28T12:38:14.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.427+0000 7fba52c00640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fba2c06c8f0 0x7fba2c06edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:14.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.427+0000 7fba52c00640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba4c161460 0x7fba4c1426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:14.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.427+0000 7fba52c00640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba4c160d90 0x7fba4c1583d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:14.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.427+0000 7fba52c00640 1 --2- 172.21.15.131:0/2393606196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba4c1463e0 0x7fba4c145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:14.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.427+0000 7fba52c00640 1 -- 172.21.15.131:0/2393606196 >> 172.21.15.131:0/2393606196 conn(0x7fba4c0b26b0 msgr2=0x7fba4c0b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:14.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.428+0000 7fba52c00640 1 -- 172.21.15.131:0/2393606196 shutdown_connections 2024-04-28T12:38:14.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:14.428+0000 7fba52c00640 1 -- 172.21.15.131:0/2393606196 wait complete. 2024-04-28T12:38:14.441 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-28T12:38:14.441 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T12:38:14.441 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-28T12:38:14.441 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-28T12:38:14.441 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-28T12:38:14.441 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-28T12:38:14.441 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-28T12:38:14.441 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T12:38:14.442 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-28T12:38:14.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:14.442 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-28T12:38:14.498 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T12:38:14.499 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T12:38:14.499 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T12:38:14.499 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:14.499 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-28T12:38:14.499 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:14.499 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:38:14.499 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:14.499 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-28T12:38:14.499 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:38:14.499 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:14.499 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42502sec preferred_lft 42502sec 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 32:3b:e7:25:fb:54 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:14.500 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5cee:75ff:fec2:82bc/64 scope link 2024-04-28T12:38:14.501 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:14.501 INFO:teuthology.orchestra.run.smithi157.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T12:38:14.501 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 32:3b:e7:25:fb:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T12:38:14.501 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::303b:e7ff:fe25:fb54/64 scope link 2024-04-28T12:38:14.501 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:14.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:14.501 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T12:38:14.579 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-28T12:38:14.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:14.584 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-28T12:38:14.637 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-28T12:38:14.637 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T12:38:14.637 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T12:38:14.637 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T12:38:14.637 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T12:38:14.638 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:14.638 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-28T12:38:14.638 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T12:38:14.638 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T12:38:14.638 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T12:38:14.638 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-28T12:38:14.745 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:14.845 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:14.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:14.855 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-28T12:38:14.905 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-28T12:38:14.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:14.906 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-04-28T12:38:14.952 INFO:teuthology.orchestra.run.smithi157.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-28T12:38:14.953 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:14.953 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-28T12:38:14.953 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-28T12:38:14.953 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-28T12:38:14.953 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-28T12:38:15.105 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:15.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:15.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:15.182 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-28T12:38:15.261 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T12:38:15.261 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T12:38:15.261 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T12:38:15.262 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:15.262 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-28T12:38:15.262 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:15.262 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T12:38:15.262 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:fc:2d:d3:99:43 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-28T12:38:15.262 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-28T12:38:15.262 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:15.262 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::cfc:2dff:fed3:9943/64 scope link 2024-04-28T12:38:15.262 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:15.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:15.265 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-28T12:38:15.342 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T12:38:15.342 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T12:38:15.342 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T12:38:15.342 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:15.342 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-28T12:38:15.342 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:15.343 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T12:38:15.343 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:fc:2d:d3:99:43 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-28T12:38:15.343 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-28T12:38:15.343 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:15.343 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::cfc:2dff:fed3:9943/64 scope link 2024-04-28T12:38:15.343 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:38:15.345 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-28T12:38:15.345 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:15.345 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-28T12:38:15.345 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-28T12:38:15.346 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-28T12:38:15.346 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-28T12:38:15.346 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-28T12:38:15.346 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-28T12:38:15.346 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-28T12:38:15.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:15.530 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:38:15.537 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:38:15.607 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:15.617 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:38:15.617 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-28T12:38:15.617 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.1 up 2024-04-28T12:38:15.617 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.1 master ceph-brx 2024-04-28T12:38:15.617 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-28T12:38:15.712 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:38:15.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:15.820 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-28T12:38:15.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:38:15.820 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-28T12:38:15.866 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-28T12:38:15.866 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:38:15.866 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-28T12:38:15.965 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-28T12:38:16.047 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:38:16.100 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-28T12:38:16.100 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-28T12:38:16.100 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-28T12:38:16.100 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-28T12:38:16.100 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-28T12:38:16.100 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-28T12:38:16.100 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-28T12:38:16.100 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-28T12:38:16.100 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-28T12:38:16.100 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-28T12:38:16.100 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:38:16.101 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-28T12:38:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-28T12:38:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-28T12:38:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-28T12:38:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-28T12:38:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-28T12:38:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-28T12:38:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T12:38:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T12:38:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-28T12:38:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-28T12:38:16.102 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-28T12:38:16.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:16.103 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-28T12:38:16.157 INFO:teuthology.orchestra.run.smithi157.stdout:40 2024-04-28T12:38:16.157 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2024-04-28T12:38:16.158 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-28T12:38:16.164 DEBUG:teuthology.orchestra.run.smithi157:> sudo modprobe fuse 2024-04-28T12:38:16.234 DEBUG:teuthology.orchestra.run.smithi157:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:38:16.277 INFO:teuthology.orchestra.run.smithi157.stdout:/proc 2024-04-28T12:38:16.277 INFO:teuthology.orchestra.run.smithi157.stdout:/sys 2024-04-28T12:38:16.277 INFO:teuthology.orchestra.run.smithi157.stdout:/dev 2024-04-28T12:38:16.277 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/security 2024-04-28T12:38:16.277 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/shm 2024-04-28T12:38:16.277 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/pts 2024-04-28T12:38:16.277 INFO:teuthology.orchestra.run.smithi157.stdout:/run 2024-04-28T12:38:16.277 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/cgroup 2024-04-28T12:38:16.277 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/pstore 2024-04-28T12:38:16.277 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/bpf 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/ 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/selinux 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/hugepages 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/dev/mqueue 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/debug 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/tracing 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/fs/fuse/connections 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/sys/kernel/config 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:38:16.278 INFO:teuthology.orchestra.run.smithi157.stdout:/run/user/1000 2024-04-28T12:38:16.279 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph 2024-04-28T12:38:16.279 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-4 2024-04-28T12:38:16.279 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-5 2024-04-28T12:38:16.279 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-6 2024-04-28T12:38:16.279 INFO:teuthology.orchestra.run.smithi157.stdout:/var/lib/ceph/osd/ceph-7 2024-04-28T12:38:16.279 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns 2024-04-28T12:38:16.279 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T12:38:16.279 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T12:38:16.279 INFO:teuthology.orchestra.run.smithi157.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-28T12:38:16.279 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-28T12:38:16.279 INFO:teuthology.orchestra.run.smithi157.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-28T12:38:16.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:16.280 DEBUG:teuthology.orchestra.run.smithi157:> ls /sys/fs/fuse/connections 2024-04-28T12:38:16.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi157.stderr:2024-04-28T12:38:16.337+0000 7f256ad1e480 -1 init, newargv = 0x5557aa7174d0 newargc=15 2024-04-28T12:38:16.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi157.stderr:ceph-fuse[38725]: starting ceph client 2024-04-28T12:38:16.344 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi157.stderr:ceph-fuse[38725]: starting fuse 2024-04-28T12:38:16.364 INFO:teuthology.orchestra.run.smithi157.stdout:40 2024-04-28T12:38:16.364 INFO:teuthology.orchestra.run.smithi157.stdout:43 2024-04-28T12:38:16.364 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 43] 2024-04-28T12:38:16.364 DEBUG:teuthology.orchestra.run.smithi157:> sudo stdin-killer -- python3 -c ' 2024-04-28T12:38:16.365 DEBUG:teuthology.orchestra.run.smithi157:> import glob 2024-04-28T12:38:16.365 DEBUG:teuthology.orchestra.run.smithi157:> import re 2024-04-28T12:38:16.365 DEBUG:teuthology.orchestra.run.smithi157:> import os 2024-04-28T12:38:16.365 DEBUG:teuthology.orchestra.run.smithi157:> import subprocess 2024-04-28T12:38:16.365 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-28T12:38:16.365 DEBUG:teuthology.orchestra.run.smithi157:> def _find_admin_socket(client_name): 2024-04-28T12:38:16.365 DEBUG:teuthology.orchestra.run.smithi157:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-28T12:38:16.365 DEBUG:teuthology.orchestra.run.smithi157:> files = glob.glob(asok_path) 2024-04-28T12:38:16.365 DEBUG:teuthology.orchestra.run.smithi157:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-28T12:38:16.365 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-28T12:38:16.365 DEBUG:teuthology.orchestra.run.smithi157:> # Given a non-glob path, it better be there 2024-04-28T12:38:16.365 DEBUG:teuthology.orchestra.run.smithi157:> if "*" not in asok_path: 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> assert(len(files) == 1) 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> return files[0] 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> for f in files: 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> contents = proc_f.read() 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> if mountpoint in contents: 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> return f 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> print(_find_admin_socket("client.3")) 2024-04-28T12:38:16.366 DEBUG:teuthology.orchestra.run.smithi157:> ' 2024-04-28T12:38:16.559 INFO:teuthology.orchestra.run.smithi157.stdout:/var/run/ceph/ceph-client.3.38725.asok 2024-04-28T12:38:16.563 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-28T12:38:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:38:16.575 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.38725.asok 2024-04-28T12:38:16.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:16.575 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.38725.asok status 2024-04-28T12:38:16.797 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-28T12:38:16.797 INFO:teuthology.orchestra.run.smithi157.stdout: "metadata": { 2024-04-28T12:38:16.797 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-28T12:38:16.797 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-28T12:38:16.797 INFO:teuthology.orchestra.run.smithi157.stdout: "entity_id": "3", 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: "hostname": "smithi157", 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: "pid": "38725", 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: "root": "/" 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_count": 0, 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: "dentry_pinned_count": 0, 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: "id": 34406, 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: "inst": { 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: "name": { 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "client", 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: "num": 34406 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-28T12:38:16.798 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.2:0", 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3048113382 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: } 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": { 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: "type": "v1", 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: "addr": "192.168.0.2:0", 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: "nonce": 3048113382 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: }, 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: "inst_str": "client.34406 192.168.0.2:0/3048113382", 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: "addr_str": "192.168.0.2:0/3048113382", 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: "inode_count": 1, 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: "mds_epoch": 9, 2024-04-28T12:38:16.799 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch": 26, 2024-04-28T12:38:16.800 INFO:teuthology.orchestra.run.smithi157.stdout: "osd_epoch_barrier": 0, 2024-04-28T12:38:16.800 INFO:teuthology.orchestra.run.smithi157.stdout: "blocklisted": false, 2024-04-28T12:38:16.800 INFO:teuthology.orchestra.run.smithi157.stdout: "fs_name": "cephfs" 2024-04-28T12:38:16.800 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-28T12:38:16.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:16.811 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-04-28T12:38:16.811 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-28T12:38:16.870 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-04-28T12:38:16.870 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-28T12:38:16.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:16.871 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-28T12:38:16.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:16.949 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-04-28T12:38:16.949 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-28T12:38:16.998 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-04-28T12:38:16.998 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-28T12:38:16.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:16.999 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-28T12:38:17.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:17.133 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-28T12:38:17.133 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-28T12:38:17.192 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-28T12:38:17.192 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-28T12:38:17.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:17.193 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-28T12:38:17.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:17.281 DEBUG:teuthology.orchestra.run.smithi157:> stat --file-system '--printf=%T 2024-04-28T12:38:17.281 DEBUG:teuthology.orchestra.run.smithi157:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-28T12:38:17.328 INFO:teuthology.orchestra.run.smithi157.stdout:fuseblk 2024-04-28T12:38:17.328 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-28T12:38:17.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:17.328 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-28T12:38:17.459 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-28T12:38:17.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:17.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.690+0000 7f1d06000640 1 Processor -- start 2024-04-28T12:38:17.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.690+0000 7f1d06000640 1 -- start start 2024-04-28T12:38:17.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.692+0000 7f1d06000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d00072620 0x7f1d000cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:17.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.692+0000 7f1d06000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d00072a80 con 0x7f1d000cd180 2024-04-28T12:38:17.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.692+0000 7f1d06000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d000cd560 con 0x7f1d000ce2b0 2024-04-28T12:38:17.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.692+0000 7f1d06000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1d000ce6e0 con 0x7f1d00072620 2024-04-28T12:38:17.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.692+0000 7f1cffe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d00072620 0x7f1d000cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:17.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.692+0000 7f1cffe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d00072620 0x7f1d000cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54788/0 (socket says 172.21.15.131:54788) 2024-04-28T12:38:17.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.692+0000 7f1d04c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1d000cd180 0x7f1d0015dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51318/0 (socket says 172.21.15.131:51318) 2024-04-28T12:38:17.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.692+0000 7f1cffe00640 1 -- 172.21.15.131:0/949094875 learned_addr learned my addr 172.21.15.131:0/949094875 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:17.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.693+0000 7f1cffe00640 1 -- 172.21.15.131:0/949094875 >> v1:172.21.15.157:6789/0 conn(0x7f1d000ce2b0 legacy=0x7f1d0006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:17.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.693+0000 7f1cffe00640 1 -- 172.21.15.131:0/949094875 >> v1:172.21.15.131:6789/0 conn(0x7f1d000cd180 legacy=0x7f1d0015dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:17.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.693+0000 7f1cffe00640 1 -- 172.21.15.131:0/949094875 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d00072d00 con 0x7f1d00072620 2024-04-28T12:38:17.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.694+0000 7f1cffe00640 1 --2- 172.21.15.131:0/949094875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d00072620 0x7f1d000cdd70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009870 tx=0x7f1cec02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47fb49e465abbc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:17.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.694+0000 7f1cff400640 1 -- 172.21.15.131:0/949094875 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec037070 con 0x7f1d00072620 2024-04-28T12:38:17.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.694+0000 7f1cff400640 1 -- 172.21.15.131:0/949094875 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cec032440 con 0x7f1d00072620 2024-04-28T12:38:17.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.694+0000 7f1cff400640 1 -- 172.21.15.131:0/949094875 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec032810 con 0x7f1d00072620 2024-04-28T12:38:17.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.695+0000 7f1d06000640 1 -- 172.21.15.131:0/949094875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d00072620 msgr2=0x7f1d000cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:17.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.695+0000 7f1d06000640 1 --2- 172.21.15.131:0/949094875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d00072620 0x7f1d000cdd70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009870 tx=0x7f1cec02a2a0 comp rx=0 tx=0).stop 2024-04-28T12:38:17.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.695+0000 7f1d06000640 1 -- 172.21.15.131:0/949094875 shutdown_connections 2024-04-28T12:38:17.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.695+0000 7f1d06000640 1 --2- 172.21.15.131:0/949094875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d00072620 0x7f1d000cdd70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:17.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.695+0000 7f1d06000640 1 -- 172.21.15.131:0/949094875 >> 172.21.15.131:0/949094875 conn(0x7f1d00068090 msgr2=0x7f1d00068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:17.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.695+0000 7f1d06000640 1 -- 172.21.15.131:0/949094875 shutdown_connections 2024-04-28T12:38:17.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.695+0000 7f1d06000640 1 -- 172.21.15.131:0/949094875 wait complete. 2024-04-28T12:38:17.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.696+0000 7f1d06000640 1 Processor -- start 2024-04-28T12:38:17.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.697+0000 7f1d06000640 1 -- start start 2024-04-28T12:38:17.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.697+0000 7f1d06000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d00072620 0x7f1d0015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.697+0000 7f1d06000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d000cd180 0x7f1d000b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.697+0000 7f1d06000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d000ce2b0 0x7f1d000b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.697+0000 7f1d06000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1d000735a0 con 0x7f1d000cd180 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.698+0000 7f1d06000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d000732c0 con 0x7f1d00072620 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.698+0000 7f1d06000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1d00073da0 con 0x7f1d000ce2b0 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.698+0000 7f1cffe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d000cd180 0x7f1d000b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.698+0000 7f1cffe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d000cd180 0x7f1d000b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54794/0 (socket says 172.21.15.131:54794) 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.698+0000 7f1cffe00640 1 -- 172.21.15.131:0/4284574761 learned_addr learned my addr 172.21.15.131:0/4284574761 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.698+0000 7f1cffe00640 1 -- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d000ce2b0 msgr2=0x7f1d000b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.698+0000 7f1cffe00640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d000ce2b0 0x7f1d000b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.698+0000 7f1cffe00640 1 -- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d00072620 msgr2=0x7f1d0015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.698+0000 7f1d04c00640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d00072620 0x7f1d0015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:17.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.698+0000 7f1cffe00640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d00072620 0x7f1d0015c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:17.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.698+0000 7f1cffe00640 1 -- 172.21.15.131:0/4284574761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d00143d40 con 0x7f1d000cd180 2024-04-28T12:38:17.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.699+0000 7f1cffe00640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d000cd180 0x7f1d000b1410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009870 tx=0x7f1cec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:17.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.699+0000 7f1cfcc00640 1 -- 172.21.15.131:0/4284574761 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec041020 con 0x7f1d000cd180 2024-04-28T12:38:17.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.699+0000 7f1cfcc00640 1 -- 172.21.15.131:0/4284574761 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cec030970 con 0x7f1d000cd180 2024-04-28T12:38:17.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.699+0000 7f1d06000640 1 -- 172.21.15.131:0/4284574761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d00144020 con 0x7f1d000cd180 2024-04-28T12:38:17.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.699+0000 7f1cfcc00640 1 -- 172.21.15.131:0/4284574761 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec037040 con 0x7f1d000cd180 2024-04-28T12:38:17.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.699+0000 7f1d06000640 1 -- 172.21.15.131:0/4284574761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d001445e0 con 0x7f1d000cd180 2024-04-28T12:38:17.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.700+0000 7f1d06000640 1 -- 172.21.15.131:0/4284574761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d00069350 con 0x7f1d000cd180 2024-04-28T12:38:17.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.701+0000 7f1cfcc00640 1 -- 172.21.15.131:0/4284574761 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cec044020 con 0x7f1d000cd180 2024-04-28T12:38:17.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.702+0000 7f1cfcc00640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1ce006c8f0 0x7f1ce006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:17.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.702+0000 7f1cfcc00640 1 -- 172.21.15.131:0/4284574761 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1cec0ac9e0 con 0x7f1d000cd180 2024-04-28T12:38:17.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.703+0000 7f1d04c00640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1ce006c8f0 0x7f1ce006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:17.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.704+0000 7f1d04c00640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1ce006c8f0 0x7f1ce006edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00012e0 tx=0x7f1cf0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:17.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.707+0000 7f1cfcc00640 1 -- 172.21.15.131:0/4284574761 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1cec04b030 con 0x7f1d000cd180 2024-04-28T12:38:17.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.916+0000 7f1d06000640 1 -- 172.21.15.131:0/4284574761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d000720e0 con 0x7f1d000cd180 2024-04-28T12:38:17.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.918+0000 7f1cfcc00640 1 -- 172.21.15.131:0/4284574761 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7f1cec07a970 con 0x7f1d000cd180 2024-04-28T12:38:17.916 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:17.916 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:17.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.921+0000 7f1d06000640 1 -- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1ce006c8f0 msgr2=0x7f1ce006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:17.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.921+0000 7f1d06000640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1ce006c8f0 0x7f1ce006edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00012e0 tx=0x7f1cf0009040 comp rx=0 tx=0).stop 2024-04-28T12:38:17.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.921+0000 7f1d06000640 1 -- 172.21.15.131:0/4284574761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d000cd180 msgr2=0x7f1d000b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.921+0000 7f1d06000640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d000cd180 0x7f1d000b1410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009870 tx=0x7f1cec0046c0 comp rx=0 tx=0).stop 2024-04-28T12:38:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.922+0000 7f1d06000640 1 -- 172.21.15.131:0/4284574761 shutdown_connections 2024-04-28T12:38:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.922+0000 7f1d06000640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1ce006c8f0 0x7f1ce006edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.922+0000 7f1d06000640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d000ce2b0 0x7f1d000b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.922+0000 7f1d06000640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d000cd180 0x7f1d000b1410 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.922+0000 7f1d06000640 1 --2- 172.21.15.131:0/4284574761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d00072620 0x7f1d0015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.922+0000 7f1d06000640 1 -- 172.21.15.131:0/4284574761 >> 172.21.15.131:0/4284574761 conn(0x7f1d00068090 msgr2=0x7f1d0006f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.922+0000 7f1d06000640 1 -- 172.21.15.131:0/4284574761 shutdown_connections 2024-04-28T12:38:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:17.922+0000 7f1d06000640 1 -- 172.21.15.131:0/4284574761 wait complete. 2024-04-28T12:38:17.937 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:17.938 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:18.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.145+0000 7fcc77e00640 1 Processor -- start 2024-04-28T12:38:18.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.145+0000 7fcc77e00640 1 -- start start 2024-04-28T12:38:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.147+0000 7fcc77e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc700cf440 0x7fcc700b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.147+0000 7fcc77e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcc700cf820 0x7fcc700cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.147+0000 7fcc77e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcc700d01c0 0x7fcc700b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.147+0000 7fcc77e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7006f420 con 0x7fcc700d01c0 2024-04-28T12:38:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.147+0000 7fcc77e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7006eaf0 con 0x7fcc700cf820 2024-04-28T12:38:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.147+0000 7fcc77e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7006edd0 con 0x7fcc700cf440 2024-04-28T12:38:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.147+0000 7fcc75e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcc700d01c0 0x7fcc700b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.147+0000 7fcc75e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcc700d01c0 0x7fcc700b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54796/0 (socket says 172.21.15.131:54796) 2024-04-28T12:38:18.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.147+0000 7fcc75e00640 1 -- 172.21.15.131:0/1458779215 learned_addr learned my addr 172.21.15.131:0/1458779215 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.148+0000 7fcc74a00640 1 --2- 172.21.15.131:0/1458779215 >> v2:172.21.15.157:3300/0 conn(0x7fcc700cf820 0x7fcc700cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.148+0000 7fcc75400640 1 --2- 172.21.15.131:0/1458779215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc700cf440 0x7fcc700b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.148+0000 7fcc75e00640 1 -- 172.21.15.131:0/1458779215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc700cf440 msgr2=0x7fcc700b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.148+0000 7fcc75e00640 1 --2- 172.21.15.131:0/1458779215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc700cf440 0x7fcc700b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.148+0000 7fcc75e00640 1 -- 172.21.15.131:0/1458779215 >> v2:172.21.15.157:3300/0 conn(0x7fcc700cf820 msgr2=0x7fcc700cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.148+0000 7fcc75e00640 1 --2- 172.21.15.131:0/1458779215 >> v2:172.21.15.157:3300/0 conn(0x7fcc700cf820 0x7fcc700cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.148+0000 7fcc75e00640 1 -- 172.21.15.131:0/1458779215 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7006f0b0 con 0x7fcc700d01c0 2024-04-28T12:38:18.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.149+0000 7fcc75e00640 1 --2- 172.21.15.131:0/1458779215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc700d01c0 0x7fcc700b41d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcc7006d7b0 tx=0x7fcc6802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a306bb2c91bc94c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.149+0000 7fcc6fe00640 1 -- 172.21.15.131:0/1458779215 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68036070 con 0x7fcc700d01c0 2024-04-28T12:38:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.149+0000 7fcc6fe00640 1 -- 172.21.15.131:0/1458779215 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc68031600 con 0x7fcc700d01c0 2024-04-28T12:38:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.149+0000 7fcc6fe00640 1 -- 172.21.15.131:0/1458779215 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc680319d0 con 0x7fcc700d01c0 2024-04-28T12:38:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.150+0000 7fcc77e00640 1 -- 172.21.15.131:0/1458779215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc700d01c0 msgr2=0x7fcc700b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.150+0000 7fcc77e00640 1 --2- 172.21.15.131:0/1458779215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc700d01c0 0x7fcc700b41d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcc7006d7b0 tx=0x7fcc6802a610 comp rx=0 tx=0).stop 2024-04-28T12:38:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.150+0000 7fcc77e00640 1 -- 172.21.15.131:0/1458779215 shutdown_connections 2024-04-28T12:38:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.150+0000 7fcc77e00640 1 --2- 172.21.15.131:0/1458779215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc700d01c0 0x7fcc700b41d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.150+0000 7fcc77e00640 1 --2- 172.21.15.131:0/1458779215 >> v2:172.21.15.157:3300/0 conn(0x7fcc700cf820 0x7fcc700cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.150+0000 7fcc77e00640 1 --2- 172.21.15.131:0/1458779215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc700cf440 0x7fcc700b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.150+0000 7fcc77e00640 1 -- 172.21.15.131:0/1458779215 >> 172.21.15.131:0/1458779215 conn(0x7fcc700b2900 msgr2=0x7fcc700b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.150+0000 7fcc77e00640 1 -- 172.21.15.131:0/1458779215 shutdown_connections 2024-04-28T12:38:18.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.150+0000 7fcc77e00640 1 -- 172.21.15.131:0/1458779215 wait complete. 2024-04-28T12:38:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.152+0000 7fcc77e00640 1 Processor -- start 2024-04-28T12:38:18.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.152+0000 7fcc77e00640 1 -- start start 2024-04-28T12:38:18.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.153+0000 7fcc77e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc700cf440 0x7fcc7015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.153+0000 7fcc77e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc700cf820 0x7fcc7015de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.153+0000 7fcc77e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc700d01c0 0x7fcc701f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.153+0000 7fcc77e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcc700b4710 con 0x7fcc700d01c0 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.153+0000 7fcc77e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7006f8b0 con 0x7fcc700cf820 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.153+0000 7fcc77e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcc700b4890 con 0x7fcc700cf440 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.153+0000 7fcc75e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc700d01c0 0x7fcc701f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.153+0000 7fcc75e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc700d01c0 0x7fcc701f48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54798/0 (socket says 172.21.15.131:54798) 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.153+0000 7fcc75e00640 1 -- 172.21.15.131:0/441177809 learned_addr learned my addr 172.21.15.131:0/441177809 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.154+0000 7fcc75400640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc700cf440 0x7fcc7015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.154+0000 7fcc74a00640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc700cf820 0x7fcc7015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.154+0000 7fcc75e00640 1 -- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc700cf440 msgr2=0x7fcc7015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.154+0000 7fcc75e00640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc700cf440 0x7fcc7015cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.154+0000 7fcc75e00640 1 -- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc700cf820 msgr2=0x7fcc7015de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:18.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.154+0000 7fcc75e00640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc700cf820 0x7fcc7015de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.154+0000 7fcc75e00640 1 -- 172.21.15.131:0/441177809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7015d060 con 0x7fcc700d01c0 2024-04-28T12:38:18.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.154+0000 7fcc75e00640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc700d01c0 0x7fcc701f48b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009f40 tx=0x7fcc68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.154+0000 7fcc6d600640 1 -- 172.21.15.131:0/441177809 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68041020 con 0x7fcc700d01c0 2024-04-28T12:38:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.154+0000 7fcc6d600640 1 -- 172.21.15.131:0/441177809 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc68007780 con 0x7fcc700d01c0 2024-04-28T12:38:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.154+0000 7fcc6d600640 1 -- 172.21.15.131:0/441177809 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68036040 con 0x7fcc700d01c0 2024-04-28T12:38:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.155+0000 7fcc77e00640 1 -- 172.21.15.131:0/441177809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc70143c70 con 0x7fcc700d01c0 2024-04-28T12:38:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.155+0000 7fcc77e00640 1 -- 172.21.15.131:0/441177809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc70144230 con 0x7fcc700d01c0 2024-04-28T12:38:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.156+0000 7fcc77e00640 1 -- 172.21.15.131:0/441177809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc70069350 con 0x7fcc700d01c0 2024-04-28T12:38:18.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.157+0000 7fcc6d600640 1 -- 172.21.15.131:0/441177809 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc68044020 con 0x7fcc700d01c0 2024-04-28T12:38:18.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.158+0000 7fcc6d600640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcc5006c8f0 0x7fcc5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.158+0000 7fcc6d600640 1 -- 172.21.15.131:0/441177809 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcc680acc90 con 0x7fcc700d01c0 2024-04-28T12:38:18.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.159+0000 7fcc75400640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcc5006c8f0 0x7fcc5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:18.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.160+0000 7fcc75400640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcc5006c8f0 0x7fcc5006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcc64006fd0 tx=0x7fcc64008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:18.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.163+0000 7fcc6d600640 1 -- 172.21.15.131:0/441177809 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcc6804b030 con 0x7fcc700d01c0 2024-04-28T12:38:18.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.366+0000 7fcc77e00640 1 -- 172.21.15.131:0/441177809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcc700affc0 con 0x7fcc700d01c0 2024-04-28T12:38:18.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.367+0000 7fcc6d600640 1 -- 172.21.15.131:0/441177809 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fcc7015d060 con 0x7fcc700d01c0 2024-04-28T12:38:18.364 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:18.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.370+0000 7fcc77e00640 1 -- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcc5006c8f0 msgr2=0x7fcc5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:18.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.370+0000 7fcc77e00640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcc5006c8f0 0x7fcc5006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcc64006fd0 tx=0x7fcc64008040 comp rx=0 tx=0).stop 2024-04-28T12:38:18.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.370+0000 7fcc77e00640 1 -- 172.21.15.131:0/441177809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc700d01c0 msgr2=0x7fcc701f48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:18.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.370+0000 7fcc77e00640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc700d01c0 0x7fcc701f48b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009f40 tx=0x7fcc68004a40 comp rx=0 tx=0).stop 2024-04-28T12:38:18.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.370+0000 7fcc77e00640 1 -- 172.21.15.131:0/441177809 shutdown_connections 2024-04-28T12:38:18.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.370+0000 7fcc77e00640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcc5006c8f0 0x7fcc5006edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.370+0000 7fcc77e00640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc700d01c0 0x7fcc701f48b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.370+0000 7fcc77e00640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc700cf820 0x7fcc7015de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.370+0000 7fcc77e00640 1 --2- 172.21.15.131:0/441177809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc700cf440 0x7fcc7015cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.370+0000 7fcc77e00640 1 -- 172.21.15.131:0/441177809 >> 172.21.15.131:0/441177809 conn(0x7fcc700b2900 msgr2=0x7fcc7006df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:18.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.371+0000 7fcc77e00640 1 -- 172.21.15.131:0/441177809 shutdown_connections 2024-04-28T12:38:18.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.371+0000 7fcc77e00640 1 -- 172.21.15.131:0/441177809 wait complete. 2024-04-28T12:38:18.386 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:18.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.599+0000 7f255cc00640 1 Processor -- start 2024-04-28T12:38:18.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.600+0000 7f255cc00640 1 -- start start 2024-04-28T12:38:18.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.601+0000 7f255cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255815f150 0x7f2558070130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.601+0000 7f255cc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f25580cd690 0x7f25580cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.601+0000 7f255cc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25580d19e0 con 0x7f25580d3610 2024-04-28T12:38:18.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.601+0000 7f255cc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f25580d0380 con 0x7f25580cd690 2024-04-28T12:38:18.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.601+0000 7f255cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f25580d1790 con 0x7f255815f150 2024-04-28T12:38:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.602+0000 7f2557e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f25580cd690 0x7f25580cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.602+0000 7f2557e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f25580cd690 0x7f25580cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54810/0 (socket says 172.21.15.131:54810) 2024-04-28T12:38:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.602+0000 7f2557e00640 1 -- 172.21.15.131:0/2034575966 learned_addr learned my addr 172.21.15.131:0/2034575966 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.602+0000 7f254fe00640 1 --1- 172.21.15.131:0/2034575966 >> v1:172.21.15.131:6789/0 conn(0x7f25580d3610 0x7f2558070670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51326/0 (socket says 172.21.15.131:51326) 2024-04-28T12:38:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.602+0000 7f2557400640 1 --2- 172.21.15.131:0/2034575966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255815f150 0x7f2558070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.602+0000 7f2557e00640 1 -- 172.21.15.131:0/2034575966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255815f150 msgr2=0x7f2558070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:18.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.602+0000 7f2557e00640 1 --2- 172.21.15.131:0/2034575966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255815f150 0x7f2558070130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.602+0000 7f2557e00640 1 -- 172.21.15.131:0/2034575966 >> v1:172.21.15.131:6789/0 conn(0x7f25580d3610 legacy=0x7f2558070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:18.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.602+0000 7f2557e00640 1 -- 172.21.15.131:0/2034575966 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25580d1cc0 con 0x7f25580cd690 2024-04-28T12:38:18.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.602+0000 7f2557400640 1 --2- 172.21.15.131:0/2034575966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255815f150 0x7f2558070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:18.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.602+0000 7f2556a00640 1 -- 172.21.15.131:0/2034575966 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 250661965 0 0) 0x7f25580d19e0 con 0x7f25580d3610 2024-04-28T12:38:18.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.603+0000 7f2557e00640 1 --2- 172.21.15.131:0/2034575966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25580cd690 0x7f25580cdaf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2550009e50 tx=0x7f255002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbd95f5a88cb6d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:18.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.604+0000 7f2556a00640 1 -- 172.21.15.131:0/2034575966 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550036070 con 0x7f25580cd690 2024-04-28T12:38:18.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.604+0000 7f2556a00640 1 -- 172.21.15.131:0/2034575966 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2550031600 con 0x7f25580cd690 2024-04-28T12:38:18.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.604+0000 7f2556a00640 1 -- 172.21.15.131:0/2034575966 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25500319d0 con 0x7f25580cd690 2024-04-28T12:38:18.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.604+0000 7f255cc00640 1 -- 172.21.15.131:0/2034575966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25580cd690 msgr2=0x7f25580cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:18.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.604+0000 7f255cc00640 1 --2- 172.21.15.131:0/2034575966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25580cd690 0x7f25580cdaf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2550009e50 tx=0x7f255002a7a0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.604+0000 7f255cc00640 1 -- 172.21.15.131:0/2034575966 shutdown_connections 2024-04-28T12:38:18.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.604+0000 7f255cc00640 1 --2- 172.21.15.131:0/2034575966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25580cd690 0x7f25580cdaf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.604+0000 7f255cc00640 1 --2- 172.21.15.131:0/2034575966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f255815f150 0x7f2558070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.604+0000 7f255cc00640 1 -- 172.21.15.131:0/2034575966 >> 172.21.15.131:0/2034575966 conn(0x7f25580d2af0 msgr2=0x7f25580d2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:18.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.604+0000 7f255cc00640 1 -- 172.21.15.131:0/2034575966 shutdown_connections 2024-04-28T12:38:18.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.604+0000 7f255cc00640 1 -- 172.21.15.131:0/2034575966 wait complete. 2024-04-28T12:38:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.606+0000 7f255cc00640 1 Processor -- start 2024-04-28T12:38:18.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.606+0000 7f255cc00640 1 -- start start 2024-04-28T12:38:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.607+0000 7f255cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25580cd690 0x7f255814fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.607+0000 7f255cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25580d3610 0x7f25580c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.607+0000 7f255cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255815f150 0x7f25580b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.607+0000 7f255cc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25580d1fa0 con 0x7f255815f150 2024-04-28T12:38:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.607+0000 7f255cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f25580cd200 con 0x7f25580cd690 2024-04-28T12:38:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.607+0000 7f255cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2558070d80 con 0x7f25580d3610 2024-04-28T12:38:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.608+0000 7f2557e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255815f150 0x7f25580b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.608+0000 7f2557e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255815f150 0x7f25580b77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54814/0 (socket says 172.21.15.131:54814) 2024-04-28T12:38:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.608+0000 7f2557e00640 1 -- 172.21.15.131:0/2970706821 learned_addr learned my addr 172.21.15.131:0/2970706821 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.608+0000 7f2557e00640 1 -- 172.21.15.131:0/2970706821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25580d3610 msgr2=0x7f25580c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.608+0000 7f2557e00640 1 --2- 172.21.15.131:0/2970706821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25580d3610 0x7f25580c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.608+0000 7f2557e00640 1 -- 172.21.15.131:0/2970706821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25580cd690 msgr2=0x7f255814fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.608+0000 7f2557e00640 1 --2- 172.21.15.131:0/2970706821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25580cd690 0x7f255814fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.608+0000 7f2557e00640 1 -- 172.21.15.131:0/2970706821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25580c8040 con 0x7f255815f150 2024-04-28T12:38:18.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.608+0000 7f2557e00640 1 --2- 172.21.15.131:0/2970706821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255815f150 0x7f25580b77b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2550034040 tx=0x7f2550004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:18.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.609+0000 7f254f400640 1 -- 172.21.15.131:0/2970706821 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550036050 con 0x7f255815f150 2024-04-28T12:38:18.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.609+0000 7f254f400640 1 -- 172.21.15.131:0/2970706821 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2550042070 con 0x7f255815f150 2024-04-28T12:38:18.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.609+0000 7f254f400640 1 -- 172.21.15.131:0/2970706821 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550009510 con 0x7f255815f150 2024-04-28T12:38:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.609+0000 7f255cc00640 1 -- 172.21.15.131:0/2970706821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25580c6fa0 con 0x7f255815f150 2024-04-28T12:38:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.609+0000 7f255cc00640 1 -- 172.21.15.131:0/2970706821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25580c7560 con 0x7f255815f150 2024-04-28T12:38:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.610+0000 7f255cc00640 1 -- 172.21.15.131:0/2970706821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2558069d70 con 0x7f255815f150 2024-04-28T12:38:18.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.611+0000 7f254f400640 1 -- 172.21.15.131:0/2970706821 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2550007bd0 con 0x7f255815f150 2024-04-28T12:38:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.612+0000 7f254f400640 1 --2- 172.21.15.131:0/2970706821 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f253406c8f0 0x7f253406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:18.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.612+0000 7f254f400640 1 -- 172.21.15.131:0/2970706821 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f25500ace90 con 0x7f255815f150 2024-04-28T12:38:18.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.612+0000 7f2557400640 1 --2- 172.21.15.131:0/2970706821 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f253406c8f0 0x7f253406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:18.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.613+0000 7f2557400640 1 --2- 172.21.15.131:0/2970706821 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f253406c8f0 0x7f253406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2548006fd0 tx=0x7f2548008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:18.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.617+0000 7f254f400640 1 -- 172.21.15.131:0/2970706821 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2558069d70 con 0x7f255815f150 2024-04-28T12:38:18.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.819+0000 7f255cc00640 1 -- 172.21.15.131:0/2970706821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f25580506c0 con 0x7f255815f150 2024-04-28T12:38:18.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.820+0000 7f254f400640 1 -- 172.21.15.131:0/2970706821 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f2550045360 con 0x7f255815f150 2024-04-28T12:38:18.816 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.822+0000 7f255cc00640 1 -- 172.21.15.131:0/2970706821 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f253406c8f0 msgr2=0x7f253406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.822+0000 7f255cc00640 1 --2- 172.21.15.131:0/2970706821 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f253406c8f0 0x7f253406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2548006fd0 tx=0x7f2548008040 comp rx=0 tx=0).stop 2024-04-28T12:38:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.822+0000 7f255cc00640 1 -- 172.21.15.131:0/2970706821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255815f150 msgr2=0x7f25580b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.822+0000 7f255cc00640 1 --2- 172.21.15.131:0/2970706821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255815f150 0x7f25580b77b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2550034040 tx=0x7f2550004a40 comp rx=0 tx=0).stop 2024-04-28T12:38:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.822+0000 7f255cc00640 1 -- 172.21.15.131:0/2970706821 shutdown_connections 2024-04-28T12:38:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.822+0000 7f255cc00640 1 --2- 172.21.15.131:0/2970706821 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f253406c8f0 0x7f253406edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.822+0000 7f255cc00640 1 --2- 172.21.15.131:0/2970706821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f255815f150 0x7f25580b77b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.822+0000 7f255cc00640 1 --2- 172.21.15.131:0/2970706821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25580d3610 0x7f25580c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.822+0000 7f255cc00640 1 --2- 172.21.15.131:0/2970706821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25580cd690 0x7f255814fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:18.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.822+0000 7f255cc00640 1 -- 172.21.15.131:0/2970706821 >> 172.21.15.131:0/2970706821 conn(0x7f25580d2af0 msgr2=0x7f25580cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:18.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.823+0000 7f255cc00640 1 -- 172.21.15.131:0/2970706821 shutdown_connections 2024-04-28T12:38:18.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:18.823+0000 7f255cc00640 1 -- 172.21.15.131:0/2970706821 wait complete. 2024-04-28T12:38:18.839 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:19.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.041+0000 7f5afb600640 1 Processor -- start 2024-04-28T12:38:19.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.041+0000 7f5afb600640 1 -- start start 2024-04-28T12:38:19.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.043+0000 7f5afb600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5af4145d80 0x7f5af4072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.043+0000 7f5afb600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5af41468c0 0x7f5af40728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.043+0000 7f5afb600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5af4146160 con 0x7f5af415d5e0 2024-04-28T12:38:19.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.043+0000 7f5afb600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5af406dfb0 con 0x7f5af41468c0 2024-04-28T12:38:19.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.043+0000 7f5afb600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5af40d0470 con 0x7f5af4145d80 2024-04-28T12:38:19.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.043+0000 7f5af9600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5af41468c0 0x7f5af40728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.043+0000 7f5af9600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5af41468c0 0x7f5af40728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54822/0 (socket says 172.21.15.131:54822) 2024-04-28T12:38:19.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.043+0000 7f5af9600640 1 -- 172.21.15.131:0/2123403491 learned_addr learned my addr 172.21.15.131:0/2123403491 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:19.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.043+0000 7f5af8c00640 1 --2- 172.21.15.131:0/2123403491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5af4145d80 0x7f5af4072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.044+0000 7f5af9600640 1 -- 172.21.15.131:0/2123403491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5af4145d80 msgr2=0x7f5af4072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:19.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.044+0000 7f5af9600640 1 --2- 172.21.15.131:0/2123403491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5af4145d80 0x7f5af4072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.044+0000 7f5af9600640 1 -- 172.21.15.131:0/2123403491 >> v1:172.21.15.157:6789/0 conn(0x7f5af415d5e0 legacy=0x7f5af406ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:19.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.044+0000 7f5af9600640 1 -- 172.21.15.131:0/2123403491 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af4074180 con 0x7f5af41468c0 2024-04-28T12:38:19.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.045+0000 7f5af9600640 1 --2- 172.21.15.131:0/2123403491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af41468c0 0x7f5af40728c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009e50 tx=0x7f5aec02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8018b5b9ac264d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:19.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.045+0000 7f5af3400640 1 -- 172.21.15.131:0/2123403491 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec036070 con 0x7f5af41468c0 2024-04-28T12:38:19.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.045+0000 7f5af3400640 1 -- 172.21.15.131:0/2123403491 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aec031600 con 0x7f5af41468c0 2024-04-28T12:38:19.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.045+0000 7f5af3400640 1 -- 172.21.15.131:0/2123403491 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0319d0 con 0x7f5af41468c0 2024-04-28T12:38:19.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.046+0000 7f5afb600640 1 -- 172.21.15.131:0/2123403491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af41468c0 msgr2=0x7f5af40728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.046+0000 7f5afb600640 1 --2- 172.21.15.131:0/2123403491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af41468c0 0x7f5af40728c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009e50 tx=0x7f5aec02a630 comp rx=0 tx=0).stop 2024-04-28T12:38:19.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.046+0000 7f5afb600640 1 -- 172.21.15.131:0/2123403491 shutdown_connections 2024-04-28T12:38:19.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.046+0000 7f5afb600640 1 --2- 172.21.15.131:0/2123403491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af41468c0 0x7f5af40728c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.046+0000 7f5afb600640 1 --2- 172.21.15.131:0/2123403491 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5af4145d80 0x7f5af4072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.046+0000 7f5afb600640 1 -- 172.21.15.131:0/2123403491 >> 172.21.15.131:0/2123403491 conn(0x7f5af40b2900 msgr2=0x7f5af40b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:19.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.046+0000 7f5afb600640 1 -- 172.21.15.131:0/2123403491 shutdown_connections 2024-04-28T12:38:19.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.046+0000 7f5afb600640 1 -- 172.21.15.131:0/2123403491 wait complete. 2024-04-28T12:38:19.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.048+0000 7f5afb600640 1 Processor -- start 2024-04-28T12:38:19.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.048+0000 7f5afb600640 1 -- start start 2024-04-28T12:38:19.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5afb600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af4145d80 0x7f5af41f0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5afb600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5af41468c0 0x7f5af413eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5afb600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af415d5e0 0x7f5af413f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5afb600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5af406e530 con 0x7f5af4145d80 2024-04-28T12:38:19.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5afb600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5af406e250 con 0x7f5af415d5e0 2024-04-28T12:38:19.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5afb600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5af40d09f0 con 0x7f5af41468c0 2024-04-28T12:38:19.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5af8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af4145d80 0x7f5af41f0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5af8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af4145d80 0x7f5af41f0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54824/0 (socket says 172.21.15.131:54824) 2024-04-28T12:38:19.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5af8c00640 1 -- 172.21.15.131:0/979043592 learned_addr learned my addr 172.21.15.131:0/979043592 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:19.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5af8c00640 1 -- 172.21.15.131:0/979043592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5af41468c0 msgr2=0x7f5af413eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5af8c00640 1 --2- 172.21.15.131:0/979043592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5af41468c0 0x7f5af413eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5af8c00640 1 -- 172.21.15.131:0/979043592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af415d5e0 msgr2=0x7f5af413f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5af8c00640 1 --2- 172.21.15.131:0/979043592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af415d5e0 0x7f5af413f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.049+0000 7f5af8c00640 1 -- 172.21.15.131:0/979043592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af413c9b0 con 0x7f5af4145d80 2024-04-28T12:38:19.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.050+0000 7f5af8c00640 1 --2- 172.21.15.131:0/979043592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af4145d80 0x7f5af41f0250 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800d5f0 tx=0x7f5ae800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:19.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.050+0000 7f5af0c00640 1 -- 172.21.15.131:0/979043592 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae80125f0 con 0x7f5af4145d80 2024-04-28T12:38:19.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.050+0000 7f5af0c00640 1 -- 172.21.15.131:0/979043592 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ae8019070 con 0x7f5af4145d80 2024-04-28T12:38:19.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.050+0000 7f5af0c00640 1 -- 172.21.15.131:0/979043592 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae80129d0 con 0x7f5af4145d80 2024-04-28T12:38:19.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.050+0000 7f5afb600640 1 -- 172.21.15.131:0/979043592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af413fa80 con 0x7f5af4145d80 2024-04-28T12:38:19.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.050+0000 7f5afb600640 1 -- 172.21.15.131:0/979043592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af413af00 con 0x7f5af4145d80 2024-04-28T12:38:19.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.051+0000 7f5afb600640 1 -- 172.21.15.131:0/979043592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af406ff50 con 0x7f5af4145d80 2024-04-28T12:38:19.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.052+0000 7f5af0c00640 1 -- 172.21.15.131:0/979043592 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ae8012b70 con 0x7f5af4145d80 2024-04-28T12:38:19.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.053+0000 7f5af0c00640 1 --2- 172.21.15.131:0/979043592 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ad406c8f0 0x7f5ad406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.053+0000 7f5af0c00640 1 -- 172.21.15.131:0/979043592 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5ae808c370 con 0x7f5af4145d80 2024-04-28T12:38:19.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.054+0000 7f5af3e00640 1 --2- 172.21.15.131:0/979043592 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ad406c8f0 0x7f5ad406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.055+0000 7f5af3e00640 1 --2- 172.21.15.131:0/979043592 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ad406c8f0 0x7f5ad406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5af4140240 tx=0x7f5ae00079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:19.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.058+0000 7f5af0c00640 1 -- 172.21.15.131:0/979043592 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5af406ff50 con 0x7f5af4145d80 2024-04-28T12:38:19.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.255+0000 7f5afb600640 1 -- 172.21.15.131:0/979043592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5af4069350 con 0x7f5af4145d80 2024-04-28T12:38:19.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.257+0000 7f5af0c00640 1 -- 172.21.15.131:0/979043592 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f5ae805a390 con 0x7f5af4145d80 2024-04-28T12:38:19.254 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.260+0000 7f5afb600640 1 -- 172.21.15.131:0/979043592 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ad406c8f0 msgr2=0x7f5ad406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.260+0000 7f5afb600640 1 --2- 172.21.15.131:0/979043592 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ad406c8f0 0x7f5ad406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5af4140240 tx=0x7f5ae00079b0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.260+0000 7f5afb600640 1 -- 172.21.15.131:0/979043592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af4145d80 msgr2=0x7f5af41f0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.260+0000 7f5afb600640 1 --2- 172.21.15.131:0/979043592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af4145d80 0x7f5af41f0250 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800d5f0 tx=0x7f5ae800dab0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.260+0000 7f5afb600640 1 -- 172.21.15.131:0/979043592 shutdown_connections 2024-04-28T12:38:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.260+0000 7f5afb600640 1 --2- 172.21.15.131:0/979043592 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ad406c8f0 0x7f5ad406edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.260+0000 7f5afb600640 1 --2- 172.21.15.131:0/979043592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5af415d5e0 0x7f5af413f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.260+0000 7f5afb600640 1 --2- 172.21.15.131:0/979043592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5af41468c0 0x7f5af413eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.260+0000 7f5afb600640 1 --2- 172.21.15.131:0/979043592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5af4145d80 0x7f5af41f0250 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.260+0000 7f5afb600640 1 -- 172.21.15.131:0/979043592 >> 172.21.15.131:0/979043592 conn(0x7f5af40b2900 msgr2=0x7f5af406e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:19.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.260+0000 7f5afb600640 1 -- 172.21.15.131:0/979043592 shutdown_connections 2024-04-28T12:38:19.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.260+0000 7f5afb600640 1 -- 172.21.15.131:0/979043592 wait complete. 2024-04-28T12:38:19.275 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:19.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.480+0000 7fa200a00640 1 Processor -- start 2024-04-28T12:38:19.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.481+0000 7fa200a00640 1 -- start start 2024-04-28T12:38:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.482+0000 7fa200a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa1fc15d4e0 0x7fa1fc06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.482+0000 7fa200a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa1fc06c890 0x7fa1fc0d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.482+0000 7fa200a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1fc06f5a0 con 0x7fa1fc146890 2024-04-28T12:38:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.482+0000 7fa200a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa1fc0d03f0 con 0x7fa1fc06c890 2024-04-28T12:38:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.482+0000 7fa200a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa1fc0700f0 con 0x7fa1fc15d4e0 2024-04-28T12:38:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.482+0000 7fa1f9e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa1fc146890 0x7fa1fc06ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51342/0 (socket says 172.21.15.131:51342) 2024-04-28T12:38:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.482+0000 7fa1f9e00640 1 -- 172.21.15.131:0/170407899 learned_addr learned my addr 172.21.15.131:0/170407899 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.482+0000 7fa1fa800640 1 --2- 172.21.15.131:0/170407899 >> v2:172.21.15.131:3300/0 conn(0x7fa1fc06c890 0x7fa1fc0d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.483+0000 7fa1f9400640 1 --2- 172.21.15.131:0/170407899 >> v2:172.21.15.157:3300/0 conn(0x7fa1fc15d4e0 0x7fa1fc06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.483+0000 7fa1fa800640 1 -- 172.21.15.131:0/170407899 >> v2:172.21.15.157:3300/0 conn(0x7fa1fc15d4e0 msgr2=0x7fa1fc06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.483+0000 7fa1fa800640 1 --2- 172.21.15.131:0/170407899 >> v2:172.21.15.157:3300/0 conn(0x7fa1fc15d4e0 0x7fa1fc06c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.483+0000 7fa1f8a00640 1 -- 172.21.15.131:0/170407899 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 686072362 0 0) 0x7fa1fc06f5a0 con 0x7fa1fc146890 2024-04-28T12:38:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.483+0000 7fa1fa800640 1 -- 172.21.15.131:0/170407899 >> v1:172.21.15.131:6789/0 conn(0x7fa1fc146890 legacy=0x7fa1fc06ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.483+0000 7fa1fa800640 1 -- 172.21.15.131:0/170407899 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc06f850 con 0x7fa1fc06c890 2024-04-28T12:38:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.483+0000 7fa1f9400640 1 --2- 172.21.15.131:0/170407899 >> v2:172.21.15.157:3300/0 conn(0x7fa1fc15d4e0 0x7fa1fc06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.484+0000 7fa1fa800640 1 --2- 172.21.15.131:0/170407899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc06c890 0x7fa1fc0d0b40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0b17e0 tx=0x7fa1f402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7c7ecd90e1876d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.484+0000 7fa1f8a00640 1 -- 172.21.15.131:0/170407899 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4007740 con 0x7fa1fc06c890 2024-04-28T12:38:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.484+0000 7fa1f8a00640 1 -- 172.21.15.131:0/170407899 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f40078e0 con 0x7fa1fc06c890 2024-04-28T12:38:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.484+0000 7fa1f8a00640 1 -- 172.21.15.131:0/170407899 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4007cb0 con 0x7fa1fc06c890 2024-04-28T12:38:19.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.485+0000 7fa200a00640 1 -- 172.21.15.131:0/170407899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc06c890 msgr2=0x7fa1fc0d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.485+0000 7fa200a00640 1 --2- 172.21.15.131:0/170407899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc06c890 0x7fa1fc0d0b40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0b17e0 tx=0x7fa1f402a680 comp rx=0 tx=0).stop 2024-04-28T12:38:19.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.485+0000 7fa200a00640 1 -- 172.21.15.131:0/170407899 shutdown_connections 2024-04-28T12:38:19.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.485+0000 7fa200a00640 1 --2- 172.21.15.131:0/170407899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc06c890 0x7fa1fc0d0b40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.485+0000 7fa200a00640 1 --2- 172.21.15.131:0/170407899 >> v2:172.21.15.157:3300/0 conn(0x7fa1fc15d4e0 0x7fa1fc06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.485+0000 7fa200a00640 1 -- 172.21.15.131:0/170407899 >> 172.21.15.131:0/170407899 conn(0x7fa1fc0b38a0 msgr2=0x7fa1fc0b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:19.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.485+0000 7fa200a00640 1 -- 172.21.15.131:0/170407899 shutdown_connections 2024-04-28T12:38:19.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.485+0000 7fa200a00640 1 -- 172.21.15.131:0/170407899 wait complete. 2024-04-28T12:38:19.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.487+0000 7fa200a00640 1 Processor -- start 2024-04-28T12:38:19.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.487+0000 7fa200a00640 1 -- start start 2024-04-28T12:38:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa200a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc06c890 0x7fa1fc1420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa200a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1fc146890 0x7fa1fc0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa200a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1fc15d4e0 0x7fa1fc0aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa200a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc06fe10 con 0x7fa1fc06c890 2024-04-28T12:38:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa200a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc0d06b0 con 0x7fa1fc15d4e0 2024-04-28T12:38:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa200a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa1fc15d940 con 0x7fa1fc146890 2024-04-28T12:38:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa1f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc06c890 0x7fa1fc1420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa1f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc06c890 0x7fa1fc1420c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54842/0 (socket says 172.21.15.131:54842) 2024-04-28T12:38:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa1f9e00640 1 -- 172.21.15.131:0/1825359324 learned_addr learned my addr 172.21.15.131:0/1825359324 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa1f9400640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1fc146890 0x7fa1fc0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa1fa800640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1fc15d4e0 0x7fa1fc0aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa1f9e00640 1 -- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1fc146890 msgr2=0x7fa1fc0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa1f9e00640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1fc146890 0x7fa1fc0cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa1f9e00640 1 -- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1fc15d4e0 msgr2=0x7fa1fc0aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.488+0000 7fa1f9e00640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1fc15d4e0 0x7fa1fc0aa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.489+0000 7fa1f9e00640 1 -- 172.21.15.131:0/1825359324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc1426e0 con 0x7fa1fc06c890 2024-04-28T12:38:19.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.489+0000 7fa1fa800640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1fc15d4e0 0x7fa1fc0aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:19.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.489+0000 7fa1f9e00640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc06c890 0x7fa1fc1420c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00c4d0 tx=0x7fa1ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:19.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.489+0000 7fa1f2000640 1 -- 172.21.15.131:0/1825359324 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec018070 con 0x7fa1fc06c890 2024-04-28T12:38:19.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.489+0000 7fa1f2000640 1 -- 172.21.15.131:0/1825359324 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ec007500 con 0x7fa1fc06c890 2024-04-28T12:38:19.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.489+0000 7fa200a00640 1 -- 172.21.15.131:0/1825359324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1fc1429c0 con 0x7fa1fc06c890 2024-04-28T12:38:19.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.489+0000 7fa1f2000640 1 -- 172.21.15.131:0/1825359324 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec004a20 con 0x7fa1fc06c890 2024-04-28T12:38:19.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.489+0000 7fa200a00640 1 -- 172.21.15.131:0/1825359324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1fc142f80 con 0x7fa1fc06c890 2024-04-28T12:38:19.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.490+0000 7fa200a00640 1 -- 172.21.15.131:0/1825359324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1fc0b7e90 con 0x7fa1fc06c890 2024-04-28T12:38:19.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.491+0000 7fa1f2000640 1 -- 172.21.15.131:0/1825359324 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1ec01b070 con 0x7fa1fc06c890 2024-04-28T12:38:19.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.492+0000 7fa1f2000640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa1d806c920 0x7fa1d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.492+0000 7fa1f2000640 1 -- 172.21.15.131:0/1825359324 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa1ec08b2e0 con 0x7fa1fc06c890 2024-04-28T12:38:19.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.493+0000 7fa1f9400640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa1d806c920 0x7fa1d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.494+0000 7fa1f9400640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa1d806c920 0x7fa1d806ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc144fb0 tx=0x7fa1e4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:19.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.497+0000 7fa1f2000640 1 -- 172.21.15.131:0/1825359324 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa1fc0b7e90 con 0x7fa1fc06c890 2024-04-28T12:38:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.695+0000 7fa200a00640 1 -- 172.21.15.131:0/1825359324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa1fc069a60 con 0x7fa1fc06c890 2024-04-28T12:38:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.696+0000 7fa1f2000640 1 -- 172.21.15.131:0/1825359324 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fa1ec059270 con 0x7fa1fc06c890 2024-04-28T12:38:19.693 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.698+0000 7fa200a00640 1 -- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa1d806c920 msgr2=0x7fa1d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.698+0000 7fa200a00640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa1d806c920 0x7fa1d806ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc144fb0 tx=0x7fa1e4008040 comp rx=0 tx=0).stop 2024-04-28T12:38:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.698+0000 7fa200a00640 1 -- 172.21.15.131:0/1825359324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc06c890 msgr2=0x7fa1fc1420c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.698+0000 7fa200a00640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc06c890 0x7fa1fc1420c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00c4d0 tx=0x7fa1ec00c990 comp rx=0 tx=0).stop 2024-04-28T12:38:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.698+0000 7fa200a00640 1 -- 172.21.15.131:0/1825359324 shutdown_connections 2024-04-28T12:38:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.698+0000 7fa200a00640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa1d806c920 0x7fa1d806ede0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.698+0000 7fa200a00640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1fc15d4e0 0x7fa1fc0aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.698+0000 7fa200a00640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1fc146890 0x7fa1fc0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.698+0000 7fa200a00640 1 --2- 172.21.15.131:0/1825359324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1fc06c890 0x7fa1fc1420c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.698+0000 7fa200a00640 1 -- 172.21.15.131:0/1825359324 >> 172.21.15.131:0/1825359324 conn(0x7fa1fc0b38a0 msgr2=0x7fa1fc0d1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:19.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.699+0000 7fa200a00640 1 -- 172.21.15.131:0/1825359324 shutdown_connections 2024-04-28T12:38:19.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.699+0000 7fa200a00640 1 -- 172.21.15.131:0/1825359324 wait complete. 2024-04-28T12:38:19.715 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:19.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.917+0000 7fbfe4a00640 1 Processor -- start 2024-04-28T12:38:19.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.918+0000 7fbfe4a00640 1 -- start start 2024-04-28T12:38:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.919+0000 7fbfe4a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbfe01628e0 0x7fbfe0162d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.919+0000 7fbfe4a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfe0145710 con 0x7fbfe00af720 2024-04-28T12:38:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.919+0000 7fbfe4a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfe0073dc0 con 0x7fbfe0145330 2024-04-28T12:38:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.919+0000 7fbfe4a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fbfe00b0250 con 0x7fbfe01628e0 2024-04-28T12:38:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.919+0000 7fbfd7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbfe01628e0 0x7fbfe0162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.920+0000 7fbfd7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbfe01628e0 0x7fbfe0162d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54858/0 (socket says 172.21.15.131:54858) 2024-04-28T12:38:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.920+0000 7fbfd7e00640 1 -- 172.21.15.131:0/2011032470 learned_addr learned my addr 172.21.15.131:0/2011032470 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:19.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.920+0000 7fbfdfe00640 1 --1- 172.21.15.131:0/2011032470 >> v1:172.21.15.131:6789/0 conn(0x7fbfe00af720 0x7fbfe00afb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51352/0 (socket says 172.21.15.131:51352) 2024-04-28T12:38:19.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.920+0000 7fbfd7e00640 1 -- 172.21.15.131:0/2011032470 >> v1:172.21.15.157:6789/0 conn(0x7fbfe0145330 legacy=0x7fbfe00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:19.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.920+0000 7fbfd7e00640 1 -- 172.21.15.131:0/2011032470 >> v1:172.21.15.131:6789/0 conn(0x7fbfe00af720 legacy=0x7fbfe00afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.920+0000 7fbfd7e00640 1 -- 172.21.15.131:0/2011032470 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe00ae960 con 0x7fbfe01628e0 2024-04-28T12:38:19.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.920+0000 7fbfdea00640 1 -- 172.21.15.131:0/2011032470 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4025600524 0 0) 0x7fbfe0145710 con 0x7fbfe00af720 2024-04-28T12:38:19.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.921+0000 7fbfd7e00640 1 --2- 172.21.15.131:0/2011032470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfe01628e0 0x7fbfe0162d70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00b03d0 tx=0x7fbfc802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120f68eb1b24e91b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:19.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.921+0000 7fbfdea00640 1 -- 172.21.15.131:0/2011032470 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8037070 con 0x7fbfe01628e0 2024-04-28T12:38:19.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.921+0000 7fbfdea00640 1 -- 172.21.15.131:0/2011032470 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc8032440 con 0x7fbfe01628e0 2024-04-28T12:38:19.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.921+0000 7fbfdea00640 1 -- 172.21.15.131:0/2011032470 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8032810 con 0x7fbfe01628e0 2024-04-28T12:38:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.922+0000 7fbfe4a00640 1 -- 172.21.15.131:0/2011032470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfe01628e0 msgr2=0x7fbfe0162d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.922+0000 7fbfe4a00640 1 --2- 172.21.15.131:0/2011032470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfe01628e0 0x7fbfe0162d70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00b03d0 tx=0x7fbfc802a290 comp rx=0 tx=0).stop 2024-04-28T12:38:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.922+0000 7fbfe4a00640 1 -- 172.21.15.131:0/2011032470 shutdown_connections 2024-04-28T12:38:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.922+0000 7fbfe4a00640 1 --2- 172.21.15.131:0/2011032470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfe01628e0 0x7fbfe0162d70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.922+0000 7fbfe4a00640 1 -- 172.21.15.131:0/2011032470 >> 172.21.15.131:0/2011032470 conn(0x7fbfe0068090 msgr2=0x7fbfe0068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.922+0000 7fbfe4a00640 1 -- 172.21.15.131:0/2011032470 shutdown_connections 2024-04-28T12:38:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.922+0000 7fbfe4a00640 1 -- 172.21.15.131:0/2011032470 wait complete. 2024-04-28T12:38:19.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.924+0000 7fbfe4a00640 1 Processor -- start 2024-04-28T12:38:19.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.924+0000 7fbfe4a00640 1 -- start start 2024-04-28T12:38:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.925+0000 7fbfe4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfe00af720 0x7fbfe01f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.925+0000 7fbfe4a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfe0145330 0x7fbfe00b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.925+0000 7fbfe4a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfe01628e0 0x7fbfe00ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.925+0000 7fbfe4a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe00af200 con 0x7fbfe00af720 2024-04-28T12:38:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.925+0000 7fbfe4a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe00aef20 con 0x7fbfe0145330 2024-04-28T12:38:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.925+0000 7fbfe4a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe0073000 con 0x7fbfe01628e0 2024-04-28T12:38:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.925+0000 7fbfdf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfe00af720 0x7fbfe01f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.925+0000 7fbfdf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfe00af720 0x7fbfe01f88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54862/0 (socket says 172.21.15.131:54862) 2024-04-28T12:38:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.925+0000 7fbfdf400640 1 -- 172.21.15.131:0/129705225 learned_addr learned my addr 172.21.15.131:0/129705225 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.926+0000 7fbfdf400640 1 -- 172.21.15.131:0/129705225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfe01628e0 msgr2=0x7fbfe00ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.926+0000 7fbfdf400640 1 --2- 172.21.15.131:0/129705225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfe01628e0 0x7fbfe00ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.926+0000 7fbfdf400640 1 -- 172.21.15.131:0/129705225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfe0145330 msgr2=0x7fbfe00b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.926+0000 7fbfdf400640 1 --2- 172.21.15.131:0/129705225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfe0145330 0x7fbfe00b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.926+0000 7fbfdf400640 1 -- 172.21.15.131:0/129705225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe00cea70 con 0x7fbfe00af720 2024-04-28T12:38:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.926+0000 7fbfdf400640 1 --2- 172.21.15.131:0/129705225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfe00af720 0x7fbfe01f88f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000c430 tx=0x7fbfd000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:19.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.926+0000 7fbfd7400640 1 -- 172.21.15.131:0/129705225 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0011790 con 0x7fbfe00af720 2024-04-28T12:38:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.926+0000 7fbfd7400640 1 -- 172.21.15.131:0/129705225 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd0017070 con 0x7fbfe00af720 2024-04-28T12:38:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.926+0000 7fbfd7400640 1 -- 172.21.15.131:0/129705225 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0011bb0 con 0x7fbfe00af720 2024-04-28T12:38:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.926+0000 7fbfe4a00640 1 -- 172.21.15.131:0/129705225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe01f7e10 con 0x7fbfe00af720 2024-04-28T12:38:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.926+0000 7fbfe4a00640 1 -- 172.21.15.131:0/129705225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe00ced50 con 0x7fbfe00af720 2024-04-28T12:38:19.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.927+0000 7fbfe4a00640 1 -- 172.21.15.131:0/129705225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe0069350 con 0x7fbfe00af720 2024-04-28T12:38:19.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.928+0000 7fbfd7400640 1 -- 172.21.15.131:0/129705225 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfd001c460 con 0x7fbfe00af720 2024-04-28T12:38:19.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.929+0000 7fbfd7400640 1 --2- 172.21.15.131:0/129705225 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbfbc06c8f0 0x7fbfbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:19.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.930+0000 7fbfd7400640 1 -- 172.21.15.131:0/129705225 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fbfd008be80 con 0x7fbfe00af720 2024-04-28T12:38:19.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.930+0000 7fbfd7e00640 1 --2- 172.21.15.131:0/129705225 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbfbc06c8f0 0x7fbfbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.931+0000 7fbfd7e00640 1 --2- 172.21.15.131:0/129705225 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbfbc06c8f0 0x7fbfbc06edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8034000 tx=0x7fbfc802c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:19.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:19.934+0000 7fbfd7400640 1 -- 172.21.15.131:0/129705225 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbfd0059e10 con 0x7fbfe00af720 2024-04-28T12:38:20.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.136+0000 7fbfe4a00640 1 -- 172.21.15.131:0/129705225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbfe015ee00 con 0x7fbfe00af720 2024-04-28T12:38:20.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.139+0000 7fbfd7400640 1 -- 172.21.15.131:0/129705225 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7fbfd0059710 con 0x7fbfe00af720 2024-04-28T12:38:20.136 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:20.136 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.141+0000 7fbfe4a00640 1 -- 172.21.15.131:0/129705225 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbfbc06c8f0 msgr2=0x7fbfbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.141+0000 7fbfe4a00640 1 --2- 172.21.15.131:0/129705225 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbfbc06c8f0 0x7fbfbc06edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8034000 tx=0x7fbfc802c000 comp rx=0 tx=0).stop 2024-04-28T12:38:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.141+0000 7fbfe4a00640 1 -- 172.21.15.131:0/129705225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfe00af720 msgr2=0x7fbfe01f88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.141+0000 7fbfe4a00640 1 --2- 172.21.15.131:0/129705225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfe00af720 0x7fbfe01f88f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000c430 tx=0x7fbfd000c8f0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.142+0000 7fbfe4a00640 1 -- 172.21.15.131:0/129705225 shutdown_connections 2024-04-28T12:38:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.142+0000 7fbfe4a00640 1 --2- 172.21.15.131:0/129705225 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbfbc06c8f0 0x7fbfbc06edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.142+0000 7fbfe4a00640 1 --2- 172.21.15.131:0/129705225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbfe01628e0 0x7fbfe00ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.142+0000 7fbfe4a00640 1 --2- 172.21.15.131:0/129705225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbfe0145330 0x7fbfe00b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.142+0000 7fbfe4a00640 1 --2- 172.21.15.131:0/129705225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfe00af720 0x7fbfe01f88f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.142+0000 7fbfe4a00640 1 -- 172.21.15.131:0/129705225 >> 172.21.15.131:0/129705225 conn(0x7fbfe0068090 msgr2=0x7fbfe00afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:20.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.142+0000 7fbfe4a00640 1 -- 172.21.15.131:0/129705225 shutdown_connections 2024-04-28T12:38:20.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.142+0000 7fbfe4a00640 1 -- 172.21.15.131:0/129705225 wait complete. 2024-04-28T12:38:20.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:20.159 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:20.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.372+0000 7fa2eca00640 1 Processor -- start 2024-04-28T12:38:20.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.372+0000 7fa2eca00640 1 -- start start 2024-04-28T12:38:20.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.373+0000 7fa2eca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa2e815a7e0 0x7fa2e80b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.374+0000 7fa2eca00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e80b3900 con 0x7fa2e80b5410 2024-04-28T12:38:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.374+0000 7fa2eca00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e80b3b70 con 0x7fa2e815d9c0 2024-04-28T12:38:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.374+0000 7fa2eca00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e815abc0 con 0x7fa2e815a7e0 2024-04-28T12:38:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.374+0000 7fa2e7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa2e815a7e0 0x7fa2e80b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.374+0000 7fa2dfe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa2e80b5410 0x7fa2e815d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51358/0 (socket says 172.21.15.131:51358) 2024-04-28T12:38:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.374+0000 7fa2dfe00640 1 -- 172.21.15.131:0/2922663841 learned_addr learned my addr 172.21.15.131:0/2922663841 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.375+0000 7fa2e7400640 1 -- 172.21.15.131:0/2922663841 >> v1:172.21.15.157:6789/0 conn(0x7fa2e815d9c0 legacy=0x7fa2e815e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:20.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.375+0000 7fa2e7400640 1 -- 172.21.15.131:0/2922663841 >> v1:172.21.15.131:6789/0 conn(0x7fa2e80b5410 legacy=0x7fa2e815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:20.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.375+0000 7fa2e7400640 1 -- 172.21.15.131:0/2922663841 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e80b3e50 con 0x7fa2e815a7e0 2024-04-28T12:38:20.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.375+0000 7fa2e6a00640 1 -- 172.21.15.131:0/2922663841 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1427982297 0 0) 0x7fa2e80b3900 con 0x7fa2e80b5410 2024-04-28T12:38:20.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.376+0000 7fa2e7400640 1 --2- 172.21.15.131:0/2922663841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815a7e0 0x7fa2e80b4650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80097a0 tx=0x7fa2d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f0de14c545a9a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:20.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.376+0000 7fa2e6a00640 1 -- 172.21.15.131:0/2922663841 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8037070 con 0x7fa2e815a7e0 2024-04-28T12:38:20.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.376+0000 7fa2e6a00640 1 -- 172.21.15.131:0/2922663841 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d8032440 con 0x7fa2e815a7e0 2024-04-28T12:38:20.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.376+0000 7fa2e6a00640 1 -- 172.21.15.131:0/2922663841 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8032810 con 0x7fa2e815a7e0 2024-04-28T12:38:20.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.376+0000 7fa2eca00640 1 -- 172.21.15.131:0/2922663841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815a7e0 msgr2=0x7fa2e80b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:20.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.376+0000 7fa2eca00640 1 --2- 172.21.15.131:0/2922663841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815a7e0 0x7fa2e80b4650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80097a0 tx=0x7fa2d802a1e0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.377+0000 7fa2eca00640 1 -- 172.21.15.131:0/2922663841 shutdown_connections 2024-04-28T12:38:20.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.377+0000 7fa2eca00640 1 --2- 172.21.15.131:0/2922663841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815a7e0 0x7fa2e80b4650 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.377+0000 7fa2eca00640 1 -- 172.21.15.131:0/2922663841 >> 172.21.15.131:0/2922663841 conn(0x7fa2e8068090 msgr2=0x7fa2e80684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:20.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.377+0000 7fa2eca00640 1 -- 172.21.15.131:0/2922663841 shutdown_connections 2024-04-28T12:38:20.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.377+0000 7fa2eca00640 1 -- 172.21.15.131:0/2922663841 wait complete. 2024-04-28T12:38:20.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.378+0000 7fa2eca00640 1 Processor -- start 2024-04-28T12:38:20.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.379+0000 7fa2eca00640 1 -- start start 2024-04-28T12:38:20.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2eca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e80b5410 0x7fa2e814c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:20.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2eca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e815a7e0 0x7fa2e8156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:20.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2eca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815d9c0 0x7fa2e81f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:20.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2eca00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e80b4b90 con 0x7fa2e815d9c0 2024-04-28T12:38:20.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2eca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e815e010 con 0x7fa2e80b5410 2024-04-28T12:38:20.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2eca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e815e9f0 con 0x7fa2e815a7e0 2024-04-28T12:38:20.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2e7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815d9c0 0x7fa2e81f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:20.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2e7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815d9c0 0x7fa2e81f7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54878/0 (socket says 172.21.15.131:54878) 2024-04-28T12:38:20.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2e7e00640 1 -- 172.21.15.131:0/552775125 learned_addr learned my addr 172.21.15.131:0/552775125 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:20.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2e7400640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e80b5410 0x7fa2e814c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:20.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2dfe00640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e815a7e0 0x7fa2e8156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:20.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2e7e00640 1 -- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e815a7e0 msgr2=0x7fa2e8156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:20.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2e7e00640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e815a7e0 0x7fa2e8156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2e7e00640 1 -- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e80b5410 msgr2=0x7fa2e814c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:20.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2e7e00640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e80b5410 0x7fa2e814c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2e7e00640 1 -- 172.21.15.131:0/552775125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e81f8750 con 0x7fa2e815d9c0 2024-04-28T12:38:20.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.380+0000 7fa2e7400640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e80b5410 0x7fa2e814c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:20.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.381+0000 7fa2e7e00640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815d9c0 0x7fa2e81f7c90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000b9a0 tx=0x7fa2e000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:20.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.381+0000 7fa2df400640 1 -- 172.21.15.131:0/552775125 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0019070 con 0x7fa2e815d9c0 2024-04-28T12:38:20.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.381+0000 7fa2df400640 1 -- 172.21.15.131:0/552775125 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e000cba0 con 0x7fa2e815d9c0 2024-04-28T12:38:20.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.381+0000 7fa2df400640 1 -- 172.21.15.131:0/552775125 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0014440 con 0x7fa2e815d9c0 2024-04-28T12:38:20.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.381+0000 7fa2eca00640 1 -- 172.21.15.131:0/552775125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e81f8a30 con 0x7fa2e815d9c0 2024-04-28T12:38:20.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.381+0000 7fa2eca00640 1 -- 172.21.15.131:0/552775125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e81562f0 con 0x7fa2e815d9c0 2024-04-28T12:38:20.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.382+0000 7fa2eca00640 1 -- 172.21.15.131:0/552775125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e81564f0 con 0x7fa2e815d9c0 2024-04-28T12:38:20.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.383+0000 7fa2df400640 1 -- 172.21.15.131:0/552775125 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2e001c070 con 0x7fa2e815d9c0 2024-04-28T12:38:20.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.384+0000 7fa2df400640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa2c406c8f0 0x7fa2c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:20.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.384+0000 7fa2df400640 1 -- 172.21.15.131:0/552775125 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa2e008bb80 con 0x7fa2e815d9c0 2024-04-28T12:38:20.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.385+0000 7fa2e7400640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa2c406c8f0 0x7fa2c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:20.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.386+0000 7fa2e7400640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa2c406c8f0 0x7fa2c406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8034000 tx=0x7fa2d802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:20.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.389+0000 7fa2df400640 1 -- 172.21.15.131:0/552775125 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa2e0059ba0 con 0x7fa2e815d9c0 2024-04-28T12:38:20.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.589+0000 7fa2eca00640 1 -- 172.21.15.131:0/552775125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa2e81535b0 con 0x7fa2e815d9c0 2024-04-28T12:38:20.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.590+0000 7fa2df400640 1 -- 172.21.15.131:0/552775125 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fa2e00594a0 con 0x7fa2e815d9c0 2024-04-28T12:38:20.587 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:20.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.592+0000 7fa2eca00640 1 -- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa2c406c8f0 msgr2=0x7fa2c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:20.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.592+0000 7fa2eca00640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa2c406c8f0 0x7fa2c406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8034000 tx=0x7fa2d802e000 comp rx=0 tx=0).stop 2024-04-28T12:38:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.592+0000 7fa2eca00640 1 -- 172.21.15.131:0/552775125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815d9c0 msgr2=0x7fa2e81f7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.592+0000 7fa2eca00640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815d9c0 0x7fa2e81f7c90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000b9a0 tx=0x7fa2e000be60 comp rx=0 tx=0).stop 2024-04-28T12:38:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.593+0000 7fa2eca00640 1 -- 172.21.15.131:0/552775125 shutdown_connections 2024-04-28T12:38:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.593+0000 7fa2eca00640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa2c406c8f0 0x7fa2c406edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.593+0000 7fa2eca00640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815d9c0 0x7fa2e81f7c90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.593+0000 7fa2eca00640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2e815a7e0 0x7fa2e8156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.593+0000 7fa2eca00640 1 --2- 172.21.15.131:0/552775125 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2e80b5410 0x7fa2e814c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.593+0000 7fa2eca00640 1 -- 172.21.15.131:0/552775125 >> 172.21.15.131:0/552775125 conn(0x7fa2e8068090 msgr2=0x7fa2e8074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.593+0000 7fa2eca00640 1 -- 172.21.15.131:0/552775125 shutdown_connections 2024-04-28T12:38:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.593+0000 7fa2eca00640 1 -- 172.21.15.131:0/552775125 wait complete. 2024-04-28T12:38:20.608 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:20.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.810+0000 7f3fff200640 1 Processor -- start 2024-04-28T12:38:20.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.811+0000 7f3fff200640 1 -- start start 2024-04-28T12:38:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.812+0000 7f3fff200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff815f180 0x7f3ff80d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.812+0000 7f3fff200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3ff80d3830 0x7f3ff80b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.812+0000 7f3fff200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff80d2d70 con 0x7f3ff80b3cb0 2024-04-28T12:38:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.812+0000 7f3fff200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff815f560 con 0x7f3ff80d3830 2024-04-28T12:38:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.812+0000 7f3fff200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff80d06e0 con 0x7f3ff815f180 2024-04-28T12:38:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.812+0000 7f3ffd400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3ff80b3cb0 0x7f3ff80d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42942/0 (socket says 172.21.15.131:42942) 2024-04-28T12:38:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.812+0000 7f3ffd400640 1 -- 172.21.15.131:0/3804871924 learned_addr learned my addr 172.21.15.131:0/3804871924 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.812+0000 7f3ffe800640 1 --2- 172.21.15.131:0/3804871924 >> v2:172.21.15.131:3300/0 conn(0x7f3ff80d3830 0x7f3ff80b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:20.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.813+0000 7f3ffde00640 1 --2- 172.21.15.131:0/3804871924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff815f180 0x7f3ff80d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:20.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.813+0000 7f3ffe800640 1 -- 172.21.15.131:0/3804871924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff815f180 msgr2=0x7f3ff80d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:20.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.813+0000 7f3ffe800640 1 --2- 172.21.15.131:0/3804871924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff815f180 0x7f3ff80d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.813+0000 7f3ffe800640 1 -- 172.21.15.131:0/3804871924 >> v1:172.21.15.131:6789/0 conn(0x7f3ff80b3cb0 legacy=0x7f3ff80d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:20.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.813+0000 7f3ffe800640 1 -- 172.21.15.131:0/3804871924 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff80d09c0 con 0x7f3ff80d3830 2024-04-28T12:38:20.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.814+0000 7f3ffe800640 1 --2- 172.21.15.131:0/3804871924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80d3830 0x7f3ff80b1740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40097a0 tx=0x7f3ff402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02d03e7d60548e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:20.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.814+0000 7f3ffca00640 1 -- 172.21.15.131:0/3804871924 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4037070 con 0x7f3ff80d3830 2024-04-28T12:38:20.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.814+0000 7f3ffca00640 1 -- 172.21.15.131:0/3804871924 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff4032440 con 0x7f3ff80d3830 2024-04-28T12:38:20.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.814+0000 7f3ffca00640 1 -- 172.21.15.131:0/3804871924 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4032810 con 0x7f3ff80d3830 2024-04-28T12:38:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.815+0000 7f3fff200640 1 -- 172.21.15.131:0/3804871924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80d3830 msgr2=0x7f3ff80b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.815+0000 7f3fff200640 1 --2- 172.21.15.131:0/3804871924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80d3830 0x7f3ff80b1740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40097a0 tx=0x7f3ff402a180 comp rx=0 tx=0).stop 2024-04-28T12:38:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.815+0000 7f3fff200640 1 -- 172.21.15.131:0/3804871924 shutdown_connections 2024-04-28T12:38:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.815+0000 7f3fff200640 1 --2- 172.21.15.131:0/3804871924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80d3830 0x7f3ff80b1740 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.815+0000 7f3fff200640 1 --2- 172.21.15.131:0/3804871924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff815f180 0x7f3ff80d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.815+0000 7f3fff200640 1 -- 172.21.15.131:0/3804871924 >> 172.21.15.131:0/3804871924 conn(0x7f3ff80d4bd0 msgr2=0x7f3ff80d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.815+0000 7f3fff200640 1 -- 172.21.15.131:0/3804871924 shutdown_connections 2024-04-28T12:38:20.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.815+0000 7f3fff200640 1 -- 172.21.15.131:0/3804871924 wait complete. 2024-04-28T12:38:20.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.817+0000 7f3fff200640 1 Processor -- start 2024-04-28T12:38:20.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.817+0000 7f3fff200640 1 -- start start 2024-04-28T12:38:20.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.818+0000 7f3fff200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff80b3cb0 0x7f3ff8152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:20.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.818+0000 7f3fff200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80d3830 0x7f3ff80cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:20.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.818+0000 7f3fff200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ff815f180 0x7f3ff80ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:20.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.818+0000 7f3fff200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff80d11c0 con 0x7f3ff80d3830 2024-04-28T12:38:20.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.818+0000 7f3fff200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff80d2a40 con 0x7f3ff815f180 2024-04-28T12:38:20.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.818+0000 7f3fff200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff8153770 con 0x7f3ff80b3cb0 2024-04-28T12:38:20.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.818+0000 7f3ffd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80d3830 0x7f3ff80cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:20.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.818+0000 7f3ffd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80d3830 0x7f3ff80cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54002/0 (socket says 172.21.15.131:54002) 2024-04-28T12:38:20.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.818+0000 7f3ffd400640 1 -- 172.21.15.131:0/2320444376 learned_addr learned my addr 172.21.15.131:0/2320444376 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:20.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.819+0000 7f3ffde00640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff80b3cb0 0x7f3ff8152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:20.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.819+0000 7f3ffd400640 1 -- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff80b3cb0 msgr2=0x7f3ff8152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:20.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.819+0000 7f3ffd400640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff80b3cb0 0x7f3ff8152c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.819+0000 7f3ffd400640 1 -- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ff815f180 msgr2=0x7f3ff80ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:20.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.819+0000 7f3ffd400640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ff815f180 0x7f3ff80ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:20.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.819+0000 7f3ffd400640 1 -- 172.21.15.131:0/2320444376 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff80b7a10 con 0x7f3ff80d3830 2024-04-28T12:38:20.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.819+0000 7f3ffd400640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80d3830 0x7f3ff80cdb80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c440 tx=0x7f3fe8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:20.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.819+0000 7f3fee000640 1 -- 172.21.15.131:0/2320444376 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8017070 con 0x7f3ff80d3830 2024-04-28T12:38:20.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.819+0000 7f3fee000640 1 -- 172.21.15.131:0/2320444376 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe8005ce0 con 0x7f3ff80d3830 2024-04-28T12:38:20.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.819+0000 7f3fee000640 1 -- 172.21.15.131:0/2320444376 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80072f0 con 0x7f3ff80d3830 2024-04-28T12:38:20.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.819+0000 7f3fff200640 1 -- 172.21.15.131:0/2320444376 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff80b7cf0 con 0x7f3ff80d3830 2024-04-28T12:38:20.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.819+0000 7f3fff200640 1 -- 172.21.15.131:0/2320444376 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff80b82b0 con 0x7f3ff80d3830 2024-04-28T12:38:20.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.821+0000 7f3fff200640 1 -- 172.21.15.131:0/2320444376 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff80506c0 con 0x7f3ff80d3830 2024-04-28T12:38:20.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.822+0000 7f3fee000640 1 -- 172.21.15.131:0/2320444376 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3fe801a070 con 0x7f3ff80d3830 2024-04-28T12:38:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.822+0000 7f3fee000640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3fdc06c8f0 0x7f3fdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.823+0000 7f3fee000640 1 -- 172.21.15.131:0/2320444376 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3fe808b260 con 0x7f3ff80d3830 2024-04-28T12:38:20.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.823+0000 7f3ffde00640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3fdc06c8f0 0x7f3fdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:20.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.824+0000 7f3ffde00640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3fdc06c8f0 0x7f3fdc06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009e90 tx=0x7f3ff000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:20.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:20.827+0000 7f3fee000640 1 -- 172.21.15.131:0/2320444376 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3fe8059320 con 0x7f3ff80d3830 2024-04-28T12:38:21.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.029+0000 7f3fff200640 1 -- 172.21.15.131:0/2320444376 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3ff80d3610 con 0x7f3ff80d3830 2024-04-28T12:38:21.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.030+0000 7f3fee000640 1 -- 172.21.15.131:0/2320444376 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f3fe8058c20 con 0x7f3ff80d3830 2024-04-28T12:38:21.026 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:21.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.032+0000 7f3fff200640 1 -- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3fdc06c8f0 msgr2=0x7f3fdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.032+0000 7f3fff200640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3fdc06c8f0 0x7f3fdc06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009e90 tx=0x7f3ff000b040 comp rx=0 tx=0).stop 2024-04-28T12:38:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.032+0000 7f3fff200640 1 -- 172.21.15.131:0/2320444376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80d3830 msgr2=0x7f3ff80cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.032+0000 7f3fff200640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80d3830 0x7f3ff80cdb80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c440 tx=0x7f3fe8001e70 comp rx=0 tx=0).stop 2024-04-28T12:38:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.032+0000 7f3fff200640 1 -- 172.21.15.131:0/2320444376 shutdown_connections 2024-04-28T12:38:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.032+0000 7f3fff200640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3fdc06c8f0 0x7f3fdc06edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.032+0000 7f3fff200640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ff815f180 0x7f3ff80ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.032+0000 7f3fff200640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ff80d3830 0x7f3ff80cdb80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.032+0000 7f3fff200640 1 --2- 172.21.15.131:0/2320444376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ff80b3cb0 0x7f3ff8152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.032+0000 7f3fff200640 1 -- 172.21.15.131:0/2320444376 >> 172.21.15.131:0/2320444376 conn(0x7f3ff80d4bd0 msgr2=0x7f3ff80b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.033+0000 7f3fff200640 1 -- 172.21.15.131:0/2320444376 shutdown_connections 2024-04-28T12:38:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.033+0000 7f3fff200640 1 -- 172.21.15.131:0/2320444376 wait complete. 2024-04-28T12:38:21.048 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:21.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.257+0000 7fce33200640 1 Processor -- start 2024-04-28T12:38:21.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.258+0000 7fce33200640 1 -- start start 2024-04-28T12:38:21.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.259+0000 7fce33200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce340d1cd0 0x7fce340d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:21.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.259+0000 7fce33200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fce340cd000 0x7fce340d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.259+0000 7fce33200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce340d0490 con 0x7fce3415f180 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.259+0000 7fce33200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fce340d1340 con 0x7fce340cd000 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.259+0000 7fce33200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fce340d0730 con 0x7fce340d1cd0 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.259+0000 7fce32800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fce340cd000 0x7fce340d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.260+0000 7fce32800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fce340cd000 0x7fce340d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54012/0 (socket says 172.21.15.131:54012) 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.259+0000 7fce31e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fce3415f180 0x7fce340d15c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42958/0 (socket says 172.21.15.131:42958) 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.260+0000 7fce32800640 1 -- 172.21.15.131:0/1405265831 learned_addr learned my addr 172.21.15.131:0/1405265831 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.260+0000 7fce29400640 1 --2- 172.21.15.131:0/1405265831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce340d1cd0 0x7fce340d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.260+0000 7fce32800640 1 -- 172.21.15.131:0/1405265831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce340d1cd0 msgr2=0x7fce340d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.260+0000 7fce32800640 1 --2- 172.21.15.131:0/1405265831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce340d1cd0 0x7fce340d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.260+0000 7fce32800640 1 -- 172.21.15.131:0/1405265831 >> v1:172.21.15.131:6789/0 conn(0x7fce3415f180 legacy=0x7fce340d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.260+0000 7fce32800640 1 -- 172.21.15.131:0/1405265831 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce340d0a10 con 0x7fce340cd000 2024-04-28T12:38:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.260+0000 7fce31400640 1 -- 172.21.15.131:0/1405265831 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024251902 0 0) 0x7fce340d0490 con 0x7fce3415f180 2024-04-28T12:38:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.261+0000 7fce32800640 1 --2- 172.21.15.131:0/1405265831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce340cd000 0x7fce340d2ed0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009e50 tx=0x7fce2c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dd16de009eeb72 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.261+0000 7fce31400640 1 -- 172.21.15.131:0/1405265831 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c036070 con 0x7fce340cd000 2024-04-28T12:38:21.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.264+0000 7fce31400640 1 -- 172.21.15.131:0/1405265831 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2c031600 con 0x7fce340cd000 2024-04-28T12:38:21.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.264+0000 7fce31400640 1 -- 172.21.15.131:0/1405265831 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0319d0 con 0x7fce340cd000 2024-04-28T12:38:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.264+0000 7fce33200640 1 -- 172.21.15.131:0/1405265831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce340cd000 msgr2=0x7fce340d2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.264+0000 7fce33200640 1 --2- 172.21.15.131:0/1405265831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce340cd000 0x7fce340d2ed0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009e50 tx=0x7fce2c02a680 comp rx=0 tx=0).stop 2024-04-28T12:38:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.265+0000 7fce33200640 1 -- 172.21.15.131:0/1405265831 shutdown_connections 2024-04-28T12:38:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.265+0000 7fce33200640 1 --2- 172.21.15.131:0/1405265831 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce340cd000 0x7fce340d2ed0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.265+0000 7fce33200640 1 --2- 172.21.15.131:0/1405265831 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce340d1cd0 0x7fce340d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.265+0000 7fce33200640 1 -- 172.21.15.131:0/1405265831 >> 172.21.15.131:0/1405265831 conn(0x7fce340d4bb0 msgr2=0x7fce340d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:21.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.265+0000 7fce33200640 1 -- 172.21.15.131:0/1405265831 shutdown_connections 2024-04-28T12:38:21.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.265+0000 7fce33200640 1 -- 172.21.15.131:0/1405265831 wait complete. 2024-04-28T12:38:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.267+0000 7fce33200640 1 Processor -- start 2024-04-28T12:38:21.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.267+0000 7fce33200640 1 -- start start 2024-04-28T12:38:21.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.268+0000 7fce33200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce340cd000 0x7fce34153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:21.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.268+0000 7fce33200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce340d1cd0 0x7fce341f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:21.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.268+0000 7fce33200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce3415f180 0x7fce341f8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.268+0000 7fce33200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fce340d57d0 con 0x7fce3415f180 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.268+0000 7fce33200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fce340d0fd0 con 0x7fce340cd000 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.268+0000 7fce33200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fce340d5950 con 0x7fce340d1cd0 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.268+0000 7fce32800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce3415f180 0x7fce341f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.268+0000 7fce32800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce3415f180 0x7fce341f8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54026/0 (socket says 172.21.15.131:54026) 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.268+0000 7fce32800640 1 -- 172.21.15.131:0/958578496 learned_addr learned my addr 172.21.15.131:0/958578496 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.269+0000 7fce29400640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce340d1cd0 0x7fce341f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.269+0000 7fce31e00640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce340cd000 0x7fce34153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.269+0000 7fce32800640 1 -- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce340d1cd0 msgr2=0x7fce341f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.269+0000 7fce32800640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce340d1cd0 0x7fce341f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.269+0000 7fce32800640 1 -- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce340cd000 msgr2=0x7fce34153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.269+0000 7fce32800640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce340cd000 0x7fce34153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.269+0000 7fce32800640 1 -- 172.21.15.131:0/958578496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce34154450 con 0x7fce3415f180 2024-04-28T12:38:21.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.269+0000 7fce32800640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce3415f180 0x7fce341f8730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fce2c034040 tx=0x7fce2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:21.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.270+0000 7fce2aa00640 1 -- 172.21.15.131:0/958578496 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c041020 con 0x7fce3415f180 2024-04-28T12:38:21.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.271+0000 7fce2aa00640 1 -- 172.21.15.131:0/958578496 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2c037040 con 0x7fce3415f180 2024-04-28T12:38:21.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.271+0000 7fce2aa00640 1 -- 172.21.15.131:0/958578496 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c036050 con 0x7fce3415f180 2024-04-28T12:38:21.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.271+0000 7fce33200640 1 -- 172.21.15.131:0/958578496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce341f8d60 con 0x7fce3415f180 2024-04-28T12:38:21.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.271+0000 7fce33200640 1 -- 172.21.15.131:0/958578496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce340a0d10 con 0x7fce3415f180 2024-04-28T12:38:21.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.272+0000 7fce33200640 1 -- 172.21.15.131:0/958578496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce340506c0 con 0x7fce3415f180 2024-04-28T12:38:21.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.273+0000 7fce2aa00640 1 -- 172.21.15.131:0/958578496 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce2c007bd0 con 0x7fce3415f180 2024-04-28T12:38:21.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.274+0000 7fce2aa00640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fce1006c8f0 0x7fce1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:21.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.274+0000 7fce2aa00640 1 -- 172.21.15.131:0/958578496 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fce2c0ac600 con 0x7fce3415f180 2024-04-28T12:38:21.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.274+0000 7fce31e00640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fce1006c8f0 0x7fce1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.275+0000 7fce31e00640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fce1006c8f0 0x7fce1006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fce24000f60 tx=0x7fce24007c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:21.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.278+0000 7fce2aa00640 1 -- 172.21.15.131:0/958578496 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce2c04b060 con 0x7fce3415f180 2024-04-28T12:38:21.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.477+0000 7fce33200640 1 -- 172.21.15.131:0/958578496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce340cd660 con 0x7fce3415f180 2024-04-28T12:38:21.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.478+0000 7fce2aa00640 1 -- 172.21.15.131:0/958578496 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fce2c07a6c0 con 0x7fce3415f180 2024-04-28T12:38:21.476 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:21.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.481+0000 7fce33200640 1 -- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fce1006c8f0 msgr2=0x7fce1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.481+0000 7fce33200640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fce1006c8f0 0x7fce1006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fce24000f60 tx=0x7fce24007c00 comp rx=0 tx=0).stop 2024-04-28T12:38:21.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.481+0000 7fce33200640 1 -- 172.21.15.131:0/958578496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce3415f180 msgr2=0x7fce341f8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.481+0000 7fce33200640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce3415f180 0x7fce341f8730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fce2c034040 tx=0x7fce2c004a40 comp rx=0 tx=0).stop 2024-04-28T12:38:21.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.481+0000 7fce33200640 1 -- 172.21.15.131:0/958578496 shutdown_connections 2024-04-28T12:38:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.481+0000 7fce33200640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fce1006c8f0 0x7fce1006edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.481+0000 7fce33200640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce3415f180 0x7fce341f8730 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.481+0000 7fce33200640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fce340d1cd0 0x7fce341f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.481+0000 7fce33200640 1 --2- 172.21.15.131:0/958578496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fce340cd000 0x7fce34153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.481+0000 7fce33200640 1 -- 172.21.15.131:0/958578496 >> 172.21.15.131:0/958578496 conn(0x7fce340d4bb0 msgr2=0x7fce340cde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.482+0000 7fce33200640 1 -- 172.21.15.131:0/958578496 shutdown_connections 2024-04-28T12:38:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.482+0000 7fce33200640 1 -- 172.21.15.131:0/958578496 wait complete. 2024-04-28T12:38:21.498 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:21.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.704+0000 7f66eb600640 1 Processor -- start 2024-04-28T12:38:21.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.704+0000 7f66eb600640 1 -- start start 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.705+0000 7f66eb600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f66e4160570 0x7f66e40d3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.705+0000 7f66eb600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f66e40b3cc0 0x7f66e40cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.705+0000 7f66eb600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66e40d1220 con 0x7f66e40cfb40 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.705+0000 7f66eb600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f66e40d1ca0 con 0x7f66e4160570 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.705+0000 7f66eb600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f66e40d0410 con 0x7f66e40b3cc0 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.706+0000 7f66e8c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f66e4160570 0x7f66e40d3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.706+0000 7f66e8c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f66e4160570 0x7f66e40d3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54028/0 (socket says 172.21.15.131:54028) 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.706+0000 7f66e8c00640 1 -- 172.21.15.131:0/3508856147 learned_addr learned my addr 172.21.15.131:0/3508856147 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.706+0000 7f66e3e00640 1 --2- 172.21.15.131:0/3508856147 >> v2:172.21.15.157:3300/0 conn(0x7f66e40b3cc0 0x7f66e40cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.706+0000 7f66e8c00640 1 -- 172.21.15.131:0/3508856147 >> v2:172.21.15.157:3300/0 conn(0x7f66e40b3cc0 msgr2=0x7f66e40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.706+0000 7f66e8c00640 1 --2- 172.21.15.131:0/3508856147 >> v2:172.21.15.157:3300/0 conn(0x7f66e40b3cc0 0x7f66e40cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.706+0000 7f66e8c00640 1 -- 172.21.15.131:0/3508856147 >> v1:172.21.15.131:6789/0 conn(0x7f66e40cfb40 legacy=0x7f66e40cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.706+0000 7f66e8c00640 1 -- 172.21.15.131:0/3508856147 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e40d14a0 con 0x7f66e4160570 2024-04-28T12:38:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.706+0000 7f66e3e00640 1 --2- 172.21.15.131:0/3508856147 >> v2:172.21.15.157:3300/0 conn(0x7f66e40b3cc0 0x7f66e40cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.707+0000 7f66e8c00640 1 --2- 172.21.15.131:0/3508856147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66e4160570 0x7f66e40d3790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f66e4073130 tx=0x7f66d402a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b862abedca83471f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.708+0000 7f66e3400640 1 -- 172.21.15.131:0/3508856147 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4036070 con 0x7f66e4160570 2024-04-28T12:38:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.708+0000 7f66e3400640 1 -- 172.21.15.131:0/3508856147 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d4031600 con 0x7f66e4160570 2024-04-28T12:38:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.708+0000 7f66e3400640 1 -- 172.21.15.131:0/3508856147 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d40319d0 con 0x7f66e4160570 2024-04-28T12:38:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.708+0000 7f66eb600640 1 -- 172.21.15.131:0/3508856147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66e4160570 msgr2=0x7f66e40d3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.708+0000 7f66eb600640 1 --2- 172.21.15.131:0/3508856147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66e4160570 0x7f66e40d3790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f66e4073130 tx=0x7f66d402a6b0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.708+0000 7f66eb600640 1 -- 172.21.15.131:0/3508856147 shutdown_connections 2024-04-28T12:38:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.708+0000 7f66eb600640 1 --2- 172.21.15.131:0/3508856147 >> v2:172.21.15.157:3300/0 conn(0x7f66e40b3cc0 0x7f66e40cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.708+0000 7f66eb600640 1 --2- 172.21.15.131:0/3508856147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66e4160570 0x7f66e40d3790 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.708+0000 7f66eb600640 1 -- 172.21.15.131:0/3508856147 >> 172.21.15.131:0/3508856147 conn(0x7f66e40d2c70 msgr2=0x7f66e40d3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.708+0000 7f66eb600640 1 -- 172.21.15.131:0/3508856147 shutdown_connections 2024-04-28T12:38:21.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.708+0000 7f66eb600640 1 -- 172.21.15.131:0/3508856147 wait complete. 2024-04-28T12:38:21.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.710+0000 7f66eb600640 1 Processor -- start 2024-04-28T12:38:21.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.710+0000 7f66eb600640 1 -- start start 2024-04-28T12:38:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.711+0000 7f66eb600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e40b3cc0 0x7f66e414d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.711+0000 7f66eb600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66e40cfb40 0x7f66e40b8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.711+0000 7f66eb600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66e4160570 0x7f66e40704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.711+0000 7f66eb600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4160ad0 con 0x7f66e40cfb40 2024-04-28T12:38:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.711+0000 7f66eb600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4160950 con 0x7f66e40b3cc0 2024-04-28T12:38:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.711+0000 7f66eb600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f66e40c77d0 con 0x7f66e4160570 2024-04-28T12:38:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.711+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66e40cfb40 0x7f66e40b8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.712+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66e40cfb40 0x7f66e40b8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54036/0 (socket says 172.21.15.131:54036) 2024-04-28T12:38:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.712+0000 7f66e3e00640 1 -- 172.21.15.131:0/4201177969 learned_addr learned my addr 172.21.15.131:0/4201177969 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.712+0000 7f66e3e00640 1 -- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66e4160570 msgr2=0x7f66e40704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.712+0000 7f66e3e00640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66e4160570 0x7f66e40704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.712+0000 7f66e3e00640 1 -- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e40b3cc0 msgr2=0x7f66e414d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.712+0000 7f66e8c00640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e40b3cc0 0x7f66e414d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:21.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.712+0000 7f66e3e00640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e40b3cc0 0x7f66e414d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.712+0000 7f66e3e00640 1 -- 172.21.15.131:0/4201177969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e414ded0 con 0x7f66e40cfb40 2024-04-28T12:38:21.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.712+0000 7f66e3e00640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66e40cfb40 0x7f66e40b8030 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c7f0 tx=0x7f66d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:21.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.713+0000 7f66e0c00640 1 -- 172.21.15.131:0/4201177969 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0015070 con 0x7f66e40cfb40 2024-04-28T12:38:21.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.713+0000 7f66e0c00640 1 -- 172.21.15.131:0/4201177969 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d0007820 con 0x7f66e40cfb40 2024-04-28T12:38:21.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.713+0000 7f66e0c00640 1 -- 172.21.15.131:0/4201177969 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0007bd0 con 0x7f66e40cfb40 2024-04-28T12:38:21.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.713+0000 7f66eb600640 1 -- 172.21.15.131:0/4201177969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e41f2f00 con 0x7f66e40cfb40 2024-04-28T12:38:21.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.713+0000 7f66eb600640 1 -- 172.21.15.131:0/4201177969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e41f34c0 con 0x7f66e40cfb40 2024-04-28T12:38:21.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.714+0000 7f66eb600640 1 -- 172.21.15.131:0/4201177969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e4069a60 con 0x7f66e40cfb40 2024-04-28T12:38:21.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.715+0000 7f66e0c00640 1 -- 172.21.15.131:0/4201177969 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66d0018070 con 0x7f66e40cfb40 2024-04-28T12:38:21.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.716+0000 7f66e0c00640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f66c406c8f0 0x7f66c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:21.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.716+0000 7f66e0c00640 1 -- 172.21.15.131:0/4201177969 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f66d008c090 con 0x7f66e40cfb40 2024-04-28T12:38:21.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.717+0000 7f66e8c00640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f66c406c8f0 0x7f66c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:21.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.718+0000 7f66e8c00640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f66c406c8f0 0x7f66c406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f66d4033000 tx=0x7f66d4009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:21.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.721+0000 7f66e0c00640 1 -- 172.21.15.131:0/4201177969 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66d005a020 con 0x7f66e40cfb40 2024-04-28T12:38:21.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.920+0000 7f66eb600640 1 -- 172.21.15.131:0/4201177969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f66e40b8b00 con 0x7f66e40cfb40 2024-04-28T12:38:21.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.921+0000 7f66e0c00640 1 -- 172.21.15.131:0/4201177969 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f66d0059920 con 0x7f66e40cfb40 2024-04-28T12:38:21.918 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:21.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.923+0000 7f66eb600640 1 -- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f66c406c8f0 msgr2=0x7f66c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.923+0000 7f66eb600640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f66c406c8f0 0x7f66c406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f66d4033000 tx=0x7f66d4009000 comp rx=0 tx=0).stop 2024-04-28T12:38:21.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.923+0000 7f66eb600640 1 -- 172.21.15.131:0/4201177969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66e40cfb40 msgr2=0x7f66e40b8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.923+0000 7f66eb600640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66e40cfb40 0x7f66e40b8030 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c7f0 tx=0x7f66d000ccb0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.924+0000 7f66eb600640 1 -- 172.21.15.131:0/4201177969 shutdown_connections 2024-04-28T12:38:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.924+0000 7f66eb600640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f66c406c8f0 0x7f66c406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.924+0000 7f66eb600640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66e4160570 0x7f66e40704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.924+0000 7f66eb600640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66e40cfb40 0x7f66e40b8030 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.924+0000 7f66eb600640 1 --2- 172.21.15.131:0/4201177969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66e40b3cc0 0x7f66e414d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.924+0000 7f66eb600640 1 -- 172.21.15.131:0/4201177969 >> 172.21.15.131:0/4201177969 conn(0x7f66e40d2c70 msgr2=0x7f66e40c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.924+0000 7f66eb600640 1 -- 172.21.15.131:0/4201177969 shutdown_connections 2024-04-28T12:38:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:21.924+0000 7f66eb600640 1 -- 172.21.15.131:0/4201177969 wait complete. 2024-04-28T12:38:21.940 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:22.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.142+0000 7ffba8600640 1 Processor -- start 2024-04-28T12:38:22.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.142+0000 7ffba8600640 1 -- start start 2024-04-28T12:38:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.143+0000 7ffba8600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffba0145590 0x7ffba00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.144+0000 7ffba8600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba006f730 0x7ffba0071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.144+0000 7ffba8600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba00cf060 con 0x7ffba00b3490 2024-04-28T12:38:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.144+0000 7ffba8600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ffba00d0100 con 0x7ffba0145590 2024-04-28T12:38:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.144+0000 7ffba8600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffba00cf340 con 0x7ffba006f730 2024-04-28T12:38:22.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.144+0000 7ffba6600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba006f730 0x7ffba0071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.144+0000 7ffba5c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffba0145590 0x7ffba00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.144+0000 7ffba5200640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7ffba00b3490 0x7ffba006f020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.131:60182/0 (socket says 172.21.15.131:60182) 2024-04-28T12:38:22.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.144+0000 7ffba5200640 1 -- 172.21.15.131:0/195075805 learned_addr learned my addr 172.21.15.131:0/195075805 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:22.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.145+0000 7ffba5c00640 1 -- 172.21.15.131:0/195075805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba006f730 msgr2=0x7ffba0071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:22.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.145+0000 7ffba5c00640 1 --2- 172.21.15.131:0/195075805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba006f730 0x7ffba0071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.145+0000 7ffba5c00640 1 -- 172.21.15.131:0/195075805 >> v1:172.21.15.157:6789/0 conn(0x7ffba00b3490 legacy=0x7ffba006f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:22.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.145+0000 7ffba5c00640 1 -- 172.21.15.131:0/195075805 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba00cf620 con 0x7ffba0145590 2024-04-28T12:38:22.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.147+0000 7ffba5c00640 1 --2- 172.21.15.131:0/195075805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba0145590 0x7ffba00ce780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb940097a0 tx=0x7ffb9402c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acc1e3fc27b8993d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:22.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.147+0000 7ffb9fe00640 1 -- 172.21.15.131:0/195075805 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94039070 con 0x7ffba0145590 2024-04-28T12:38:22.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.147+0000 7ffb9fe00640 1 -- 172.21.15.131:0/195075805 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb94034640 con 0x7ffba0145590 2024-04-28T12:38:22.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.147+0000 7ffb9fe00640 1 -- 172.21.15.131:0/195075805 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94034a10 con 0x7ffba0145590 2024-04-28T12:38:22.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.148+0000 7ffba8600640 1 -- 172.21.15.131:0/195075805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba0145590 msgr2=0x7ffba00ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:22.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.148+0000 7ffba8600640 1 --2- 172.21.15.131:0/195075805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba0145590 0x7ffba00ce780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb940097a0 tx=0x7ffb9402c390 comp rx=0 tx=0).stop 2024-04-28T12:38:22.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.148+0000 7ffba8600640 1 -- 172.21.15.131:0/195075805 shutdown_connections 2024-04-28T12:38:22.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.148+0000 7ffba8600640 1 --2- 172.21.15.131:0/195075805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba006f730 0x7ffba0071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.148+0000 7ffba8600640 1 --2- 172.21.15.131:0/195075805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba0145590 0x7ffba00ce780 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.148+0000 7ffba8600640 1 -- 172.21.15.131:0/195075805 >> 172.21.15.131:0/195075805 conn(0x7ffba00b2900 msgr2=0x7ffba00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:22.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.148+0000 7ffba8600640 1 -- 172.21.15.131:0/195075805 shutdown_connections 2024-04-28T12:38:22.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.148+0000 7ffba8600640 1 -- 172.21.15.131:0/195075805 wait complete. 2024-04-28T12:38:22.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.151+0000 7ffba8600640 1 Processor -- start 2024-04-28T12:38:22.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.151+0000 7ffba8600640 1 -- start start 2024-04-28T12:38:22.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.152+0000 7ffba8600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba006f730 0x7ffba015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.152+0000 7ffba8600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba00b3490 0x7ffba015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.152+0000 7ffba8600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba0145590 0x7ffba013c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.152+0000 7ffba8600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0145970 con 0x7ffba00b3490 2024-04-28T12:38:22.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.152+0000 7ffba8600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffba00cfbe0 con 0x7ffba006f730 2024-04-28T12:38:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.152+0000 7ffba8600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffba00cecc0 con 0x7ffba0145590 2024-04-28T12:38:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.152+0000 7ffba5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba00b3490 0x7ffba015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.153+0000 7ffba5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba00b3490 0x7ffba015cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54044/0 (socket says 172.21.15.131:54044) 2024-04-28T12:38:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.153+0000 7ffba5200640 1 -- 172.21.15.131:0/1575984893 learned_addr learned my addr 172.21.15.131:0/1575984893 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.153+0000 7ffba5c00640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba006f730 0x7ffba015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.153+0000 7ffba6600640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba0145590 0x7ffba013c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.153+0000 7ffba5200640 1 -- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba0145590 msgr2=0x7ffba013c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:22.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.153+0000 7ffba5200640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba0145590 0x7ffba013c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.153+0000 7ffba5200640 1 -- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba006f730 msgr2=0x7ffba015b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:22.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.153+0000 7ffba5200640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba006f730 0x7ffba015b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.153+0000 7ffba5200640 1 -- 172.21.15.131:0/1575984893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba015bc50 con 0x7ffba00b3490 2024-04-28T12:38:22.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.153+0000 7ffba5200640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba00b3490 0x7ffba015cde0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00c440 tx=0x7ffb8c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.154+0000 7ffb9d600640 1 -- 172.21.15.131:0/1575984893 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c017070 con 0x7ffba00b3490 2024-04-28T12:38:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.154+0000 7ffb9d600640 1 -- 172.21.15.131:0/1575984893 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c005ce0 con 0x7ffba00b3490 2024-04-28T12:38:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.154+0000 7ffb9d600640 1 -- 172.21.15.131:0/1575984893 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0072f0 con 0x7ffba00b3490 2024-04-28T12:38:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.154+0000 7ffba8600640 1 -- 172.21.15.131:0/1575984893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba015d6e0 con 0x7ffba00b3490 2024-04-28T12:38:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.154+0000 7ffba8600640 1 -- 172.21.15.131:0/1575984893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba015dca0 con 0x7ffba00b3490 2024-04-28T12:38:22.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.155+0000 7ffba8600640 1 -- 172.21.15.131:0/1575984893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba0069350 con 0x7ffba00b3490 2024-04-28T12:38:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.156+0000 7ffb9d600640 1 -- 172.21.15.131:0/1575984893 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb8c01a070 con 0x7ffba00b3490 2024-04-28T12:38:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.157+0000 7ffb9d600640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffb8006c8f0 0x7ffb8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.157+0000 7ffb9d600640 1 -- 172.21.15.131:0/1575984893 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ffb8c08b310 con 0x7ffba00b3490 2024-04-28T12:38:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.158+0000 7ffba5c00640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffb8006c8f0 0x7ffb8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.159+0000 7ffba5c00640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffb8006c8f0 0x7ffb8006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffb94036000 tx=0x7ffb94030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:22.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.162+0000 7ffb9d600640 1 -- 172.21.15.131:0/1575984893 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffb8c059330 con 0x7ffba00b3490 2024-04-28T12:38:22.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.358+0000 7ffba8600640 1 -- 172.21.15.131:0/1575984893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffba00750e0 con 0x7ffba00b3490 2024-04-28T12:38:22.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.361+0000 7ffb9d600640 1 -- 172.21.15.131:0/1575984893 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7ffb8c058c30 con 0x7ffba00b3490 2024-04-28T12:38:22.358 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:22.358 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:22.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.364+0000 7ffba8600640 1 -- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffb8006c8f0 msgr2=0x7ffb8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:22.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.364+0000 7ffba8600640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffb8006c8f0 0x7ffb8006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffb94036000 tx=0x7ffb94030000 comp rx=0 tx=0).stop 2024-04-28T12:38:22.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.364+0000 7ffba8600640 1 -- 172.21.15.131:0/1575984893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba00b3490 msgr2=0x7ffba015cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:22.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.364+0000 7ffba8600640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba00b3490 0x7ffba015cde0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00c440 tx=0x7ffb8c001e70 comp rx=0 tx=0).stop 2024-04-28T12:38:22.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.364+0000 7ffba8600640 1 -- 172.21.15.131:0/1575984893 shutdown_connections 2024-04-28T12:38:22.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.364+0000 7ffba8600640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffb8006c8f0 0x7ffb8006edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.364+0000 7ffba8600640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffba0145590 0x7ffba013c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.364+0000 7ffba8600640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba00b3490 0x7ffba015cde0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.364+0000 7ffba8600640 1 --2- 172.21.15.131:0/1575984893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffba006f730 0x7ffba015b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.364+0000 7ffba8600640 1 -- 172.21.15.131:0/1575984893 >> 172.21.15.131:0/1575984893 conn(0x7ffba00b2900 msgr2=0x7ffba00b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:22.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.364+0000 7ffba8600640 1 -- 172.21.15.131:0/1575984893 shutdown_connections 2024-04-28T12:38:22.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.364+0000 7ffba8600640 1 -- 172.21.15.131:0/1575984893 wait complete. 2024-04-28T12:38:22.379 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:22.380 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:22.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.578+0000 7f5551e00640 1 Processor -- start 2024-04-28T12:38:22.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.578+0000 7f5551e00640 1 -- start start 2024-04-28T12:38:22.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.580+0000 7f5551e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f554c0b5b10 0x7f554c0b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.580+0000 7f5551e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f554c0b4fc0 con 0x7f554c155cd0 2024-04-28T12:38:22.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.580+0000 7f5551e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f554c15f920 con 0x7f554c15f3f0 2024-04-28T12:38:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.580+0000 7f5551e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f554c15fc00 con 0x7f554c0b5b10 2024-04-28T12:38:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.580+0000 7f5550a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f554c155cd0 0x7f554c072600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42976/0 (socket says 172.21.15.131:42976) 2024-04-28T12:38:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.580+0000 7f5550a00640 1 -- 172.21.15.131:0/3611972015 learned_addr learned my addr 172.21.15.131:0/3611972015 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.580+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 826049703 0 0) 0x7f554c0b4fc0 con 0x7f554c155cd0 2024-04-28T12:38:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.580+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f552c003520 con 0x7f554c155cd0 2024-04-28T12:38:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.580+0000 7f5551400640 1 --2- 172.21.15.131:0/3611972015 >> v2:172.21.15.157:3300/0 conn(0x7f554c0b5b10 0x7f554c0b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3748691310 0 0) 0x7f552c003520 con 0x7f554c155cd0 2024-04-28T12:38:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 >> v2:172.21.15.157:3300/0 conn(0x7f554c0b5b10 msgr2=0x7f554c0b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 --2- 172.21.15.131:0/3611972015 >> v2:172.21.15.157:3300/0 conn(0x7f554c0b5b10 0x7f554c0b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 >> v1:172.21.15.157:6789/0 conn(0x7f554c15f3f0 legacy=0x7f554c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c15fee0 con 0x7f554c155cd0 2024-04-28T12:38:22.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f5534001670 con 0x7f554c155cd0 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 >> v1:172.21.15.131:6789/0 conn(0x7f554c155cd0 legacy=0x7f554c072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 --2- 172.21.15.131:0/3611972015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f552c003a70 0x7f552c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 --2- 172.21.15.131:0/3611972015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 0x7f552c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 --2- 172.21.15.131:0/3611972015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f552c0052e0 0x7f552c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f554c15fc00 con 0x7f552c004800 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f552c00c1a0 con 0x7f552c003a70 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f552c00c3a0 con 0x7f552c0052e0 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f5543e00640 1 --2- 172.21.15.131:0/3611972015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 0x7f552c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f5550a00640 1 --2- 172.21.15.131:0/3611972015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f552c003a70 0x7f552c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f5543e00640 1 -- 172.21.15.131:0/3611972015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f552c0052e0 msgr2=0x7f552c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.581+0000 7f5551400640 1 --2- 172.21.15.131:0/3611972015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f552c0052e0 0x7f552c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.582+0000 7f5543e00640 1 --2- 172.21.15.131:0/3611972015 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f552c0052e0 0x7f552c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.582+0000 7f5543e00640 1 -- 172.21.15.131:0/3611972015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f552c003a70 msgr2=0x7f552c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:22.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.582+0000 7f5543e00640 1 --2- 172.21.15.131:0/3611972015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f552c003a70 0x7f552c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.582+0000 7f5543e00640 1 -- 172.21.15.131:0/3611972015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c003520 con 0x7f552c004800 2024-04-28T12:38:22.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.582+0000 7f5551400640 1 -- 172.21.15.131:0/3611972015 reap_dead start 2024-04-28T12:38:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.583+0000 7f5543e00640 1 --2- 172.21.15.131:0/3611972015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 0x7f552c004c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f553c003d80 tx=0x7f552c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e94ec3acbf18c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.583+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c037070 con 0x7f552c004800 2024-04-28T12:38:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.583+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c02a9c0 con 0x7f552c004800 2024-04-28T12:38:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.583+0000 7f554be00640 1 -- 172.21.15.131:0/3611972015 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c003980 con 0x7f552c004800 2024-04-28T12:38:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.583+0000 7f5551e00640 1 -- 172.21.15.131:0/3611972015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 msgr2=0x7f552c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.583+0000 7f5551e00640 1 --2- 172.21.15.131:0/3611972015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 0x7f552c004c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f553c003d80 tx=0x7f552c005830 comp rx=0 tx=0).stop 2024-04-28T12:38:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.583+0000 7f5551e00640 1 -- 172.21.15.131:0/3611972015 shutdown_connections 2024-04-28T12:38:22.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.583+0000 7f5551e00640 1 --2- 172.21.15.131:0/3611972015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 0x7f552c004c80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.583+0000 7f5551e00640 1 -- 172.21.15.131:0/3611972015 >> 172.21.15.131:0/3611972015 conn(0x7f554c068090 msgr2=0x7f554c0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:22.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.583+0000 7f5551e00640 1 -- 172.21.15.131:0/3611972015 shutdown_connections 2024-04-28T12:38:22.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.583+0000 7f5551e00640 1 -- 172.21.15.131:0/3611972015 wait complete. 2024-04-28T12:38:22.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.585+0000 7f5551e00640 1 Processor -- start 2024-04-28T12:38:22.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.585+0000 7f5551e00640 1 -- start start 2024-04-28T12:38:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.586+0000 7f5551e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 0x7f554c156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.586+0000 7f5551e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f554c13f580 0x7f554c1f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.586+0000 7f5551e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f554c1f3df0 0x7f554c1f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.586+0000 7f5551e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f554c1560b0 con 0x7f552c004800 2024-04-28T12:38:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.586+0000 7f5551e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f554c1629e0 con 0x7f554c1f3df0 2024-04-28T12:38:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.586+0000 7f5551e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f554c156230 con 0x7f554c13f580 2024-04-28T12:38:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.586+0000 7f5550a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 0x7f554c156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.586+0000 7f5550a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 0x7f554c156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54056/0 (socket says 172.21.15.131:54056) 2024-04-28T12:38:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.586+0000 7f5550a00640 1 -- 172.21.15.131:0/281528887 learned_addr learned my addr 172.21.15.131:0/281528887 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.587+0000 7f5550a00640 1 -- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f554c13f580 msgr2=0x7f554c1f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:22.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.587+0000 7f5550a00640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f554c13f580 0x7f554c1f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.587+0000 7f5550a00640 1 -- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f554c1f3df0 msgr2=0x7f554c1f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:22.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.587+0000 7f5551400640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f554c1f3df0 0x7f554c1f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.587+0000 7f5550a00640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f554c1f3df0 0x7f554c1f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.587+0000 7f5550a00640 1 -- 172.21.15.131:0/281528887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c158c30 con 0x7f552c004800 2024-04-28T12:38:22.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.587+0000 7f5550a00640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 0x7f554c156410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f553400e700 tx=0x7f553400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:22.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.587+0000 7f5549600640 1 -- 172.21.15.131:0/281528887 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553400c660 con 0x7f552c004800 2024-04-28T12:38:22.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.587+0000 7f5549600640 1 -- 172.21.15.131:0/281528887 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5534018070 con 0x7f552c004800 2024-04-28T12:38:22.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.587+0000 7f5551e00640 1 -- 172.21.15.131:0/281528887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554c158f10 con 0x7f552c004800 2024-04-28T12:38:22.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.587+0000 7f5549600640 1 -- 172.21.15.131:0/281528887 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553400ca20 con 0x7f552c004800 2024-04-28T12:38:22.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.587+0000 7f5551e00640 1 -- 172.21.15.131:0/281528887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554c13f9e0 con 0x7f552c004800 2024-04-28T12:38:22.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.588+0000 7f5551e00640 1 -- 172.21.15.131:0/281528887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554c0506c0 con 0x7f552c004800 2024-04-28T12:38:22.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.589+0000 7f5549600640 1 -- 172.21.15.131:0/281528887 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f553400cbc0 con 0x7f552c004800 2024-04-28T12:38:22.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.590+0000 7f5549600640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f552806c920 0x7f552806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.590+0000 7f5549600640 1 -- 172.21.15.131:0/281528887 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f553408ba10 con 0x7f552c004800 2024-04-28T12:38:22.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.591+0000 7f5543e00640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f552806c920 0x7f552806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.592+0000 7f5543e00640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f552806c920 0x7f552806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f553c034000 tx=0x7f553c02c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:22.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.595+0000 7f5549600640 1 -- 172.21.15.131:0/281528887 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f55340599a0 con 0x7f552c004800 2024-04-28T12:38:22.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.780+0000 7f5551e00640 1 -- 172.21.15.131:0/281528887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f554c155440 con 0x7f552c004800 2024-04-28T12:38:22.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.781+0000 7f5549600640 1 -- 172.21.15.131:0/281528887 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f55340592a0 con 0x7f552c004800 2024-04-28T12:38:22.777 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:22.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.783+0000 7f5551e00640 1 -- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f552806c920 msgr2=0x7f552806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:22.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.783+0000 7f5551e00640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f552806c920 0x7f552806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f553c034000 tx=0x7f553c02c000 comp rx=0 tx=0).stop 2024-04-28T12:38:22.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.783+0000 7f5551e00640 1 -- 172.21.15.131:0/281528887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 msgr2=0x7f554c156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:22.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.783+0000 7f5551e00640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 0x7f554c156410 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f553400e700 tx=0x7f553400ebc0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.783+0000 7f5551e00640 1 -- 172.21.15.131:0/281528887 shutdown_connections 2024-04-28T12:38:22.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.783+0000 7f5551e00640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f552806c920 0x7f552806ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.783+0000 7f5551e00640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f554c1f3df0 0x7f554c1f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.783+0000 7f5551e00640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f554c13f580 0x7f554c1f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.783+0000 7f5551e00640 1 --2- 172.21.15.131:0/281528887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f552c004800 0x7f554c156410 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:22.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.783+0000 7f5551e00640 1 -- 172.21.15.131:0/281528887 >> 172.21.15.131:0/281528887 conn(0x7f554c068090 msgr2=0x7f554c06e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:22.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.783+0000 7f5551e00640 1 -- 172.21.15.131:0/281528887 shutdown_connections 2024-04-28T12:38:22.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.783+0000 7f5551e00640 1 -- 172.21.15.131:0/281528887 wait complete. 2024-04-28T12:38:22.797 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:22.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.998+0000 7f7a8ec00640 1 Processor -- start 2024-04-28T12:38:22.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:22.999+0000 7f7a8ec00640 1 -- start start 2024-04-28T12:38:22.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.000+0000 7f7a8ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815f180 0x7f7a8806c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.000+0000 7f7a8ec00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7a880b4ba0 0x7f7a8815fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:22.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.000+0000 7f7a8ec00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8806caa0 con 0x7f7a88160400 2024-04-28T12:38:22.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.000+0000 7f7a8ec00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7a8806cd80 con 0x7f7a880b4ba0 2024-04-28T12:38:22.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.000+0000 7f7a8ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8806d060 con 0x7f7a8815f180 2024-04-28T12:38:22.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a8cc00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7a88160400 0x7f7a880b0b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42980/0 (socket says 172.21.15.131:42980) 2024-04-28T12:38:22.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a8cc00640 1 -- 172.21.15.131:0/42570248 learned_addr learned my addr 172.21.15.131:0/42570248 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:22.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a87400640 1 --2- 172.21.15.131:0/42570248 >> v2:172.21.15.157:3300/0 conn(0x7f7a880b4ba0 0x7f7a8815fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a87e00640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815f180 0x7f7a8806c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:22.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855663779 0 0) 0x7f7a8806caa0 con 0x7f7a88160400 2024-04-28T12:38:22.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a6c003520 con 0x7f7a88160400 2024-04-28T12:38:22.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3840167738 0 0) 0x7f7a6c003520 con 0x7f7a88160400 2024-04-28T12:38:22.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815f180 msgr2=0x7f7a8806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a86a00640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815f180 0x7f7a8806c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 >> v2:172.21.15.157:3300/0 conn(0x7f7a880b4ba0 msgr2=0x7f7a8815fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a86a00640 1 --2- 172.21.15.131:0/42570248 >> v2:172.21.15.157:3300/0 conn(0x7f7a880b4ba0 0x7f7a8815fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8815f9a0 con 0x7f7a88160400 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f7a80001670 con 0x7f7a88160400 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a87e00640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815f180 0x7f7a8806c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 >> v1:172.21.15.131:6789/0 conn(0x7f7a88160400 legacy=0x7f7a880b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.001+0000 7f7a86a00640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a6c0039b0 0x7f7a6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a86a00640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c004800 0x7f7a6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a86a00640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815f180 0x7f7a6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8806cd80 con 0x7f7a6c004800 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8806d060 con 0x7f7a6c0039b0 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7a6c00ba70 con 0x7f7a8815f180 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a87400640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c004800 0x7f7a6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a87e00640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a6c0039b0 0x7f7a6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a87400640 1 -- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815f180 msgr2=0x7f7a6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a8cc00640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815f180 0x7f7a6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a87400640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815f180 0x7f7a6c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a87400640 1 -- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a6c0039b0 msgr2=0x7f7a6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a87400640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a6c0039b0 0x7f7a6c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a87400640 1 -- 172.21.15.131:0/42570248 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c003520 con 0x7f7a6c004800 2024-04-28T12:38:23.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.002+0000 7f7a87e00640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a6c0039b0 0x7f7a6c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.003+0000 7f7a87400640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c004800 0x7f7a6c004c80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009f90 tx=0x7f7a6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf0f2fbffffee67 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.004+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78009c10 con 0x7f7a6c004800 2024-04-28T12:38:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.004+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a78036070 con 0x7f7a6c004800 2024-04-28T12:38:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.004+0000 7f7a86a00640 1 -- 172.21.15.131:0/42570248 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78031850 con 0x7f7a6c004800 2024-04-28T12:38:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.004+0000 7f7a8ec00640 1 -- 172.21.15.131:0/42570248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c004800 msgr2=0x7f7a6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.004+0000 7f7a8ec00640 1 --2- 172.21.15.131:0/42570248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c004800 0x7f7a6c004c80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009f90 tx=0x7f7a6c005830 comp rx=0 tx=0).stop 2024-04-28T12:38:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.004+0000 7f7a8cc00640 1 -- 172.21.15.131:0/42570248 reap_dead start 2024-04-28T12:38:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.004+0000 7f7a8ec00640 1 -- 172.21.15.131:0/42570248 shutdown_connections 2024-04-28T12:38:23.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.004+0000 7f7a8ec00640 1 -- 172.21.15.131:0/42570248 >> 172.21.15.131:0/42570248 conn(0x7f7a880b38a0 msgr2=0x7f7a880b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:23.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.004+0000 7f7a8ec00640 1 -- 172.21.15.131:0/42570248 shutdown_connections 2024-04-28T12:38:23.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.004+0000 7f7a8ec00640 1 -- 172.21.15.131:0/42570248 wait complete. 2024-04-28T12:38:23.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.006+0000 7f7a8ec00640 1 Processor -- start 2024-04-28T12:38:23.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.006+0000 7f7a8ec00640 1 -- start start 2024-04-28T12:38:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.007+0000 7f7a8ec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c004800 0x7f7a880b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.007+0000 7f7a8ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8814dc50 0x7f7a880af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.007+0000 7f7a8ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815c770 0x7f7a880afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.007+0000 7f7a8ec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7a880b12a0 con 0x7f7a6c004800 2024-04-28T12:38:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.007+0000 7f7a8ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7a880b1420 con 0x7f7a8814dc50 2024-04-28T12:38:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.007+0000 7f7a8ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8806c350 con 0x7f7a8815c770 2024-04-28T12:38:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.008+0000 7f7a87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c004800 0x7f7a880b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.008+0000 7f7a87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c004800 0x7f7a880b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54062/0 (socket says 172.21.15.131:54062) 2024-04-28T12:38:23.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.008+0000 7f7a87e00640 1 -- 172.21.15.131:0/2352006874 learned_addr learned my addr 172.21.15.131:0/2352006874 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:23.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.008+0000 7f7a87400640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8814dc50 0x7f7a880af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.008+0000 7f7a87e00640 1 -- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815c770 msgr2=0x7f7a880afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.008+0000 7f7a8cc00640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815c770 0x7f7a880afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.008+0000 7f7a87e00640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815c770 0x7f7a880afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.008+0000 7f7a87e00640 1 -- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8814dc50 msgr2=0x7f7a880af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.008+0000 7f7a87e00640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8814dc50 0x7f7a880af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.008+0000 7f7a87e00640 1 -- 172.21.15.131:0/2352006874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a881516c0 con 0x7f7a6c004800 2024-04-28T12:38:23.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.008+0000 7f7a87400640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8814dc50 0x7f7a880af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:23.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.008+0000 7f7a87e00640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c004800 0x7f7a880b1a70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000f650 tx=0x7f7a7000fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:23.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.009+0000 7f7a7fe00640 1 -- 172.21.15.131:0/2352006874 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a70018070 con 0x7f7a6c004800 2024-04-28T12:38:23.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.009+0000 7f7a7fe00640 1 -- 172.21.15.131:0/2352006874 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a700098e0 con 0x7f7a6c004800 2024-04-28T12:38:23.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.009+0000 7f7a7fe00640 1 -- 172.21.15.131:0/2352006874 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a70009cd0 con 0x7f7a6c004800 2024-04-28T12:38:23.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.009+0000 7f7a8ec00640 1 -- 172.21.15.131:0/2352006874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a880b22c0 con 0x7f7a6c004800 2024-04-28T12:38:23.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.009+0000 7f7a8ec00640 1 -- 172.21.15.131:0/2352006874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a880b2880 con 0x7f7a6c004800 2024-04-28T12:38:23.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.010+0000 7f7a8ec00640 1 -- 172.21.15.131:0/2352006874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a88069a60 con 0x7f7a6c004800 2024-04-28T12:38:23.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.011+0000 7f7a7fe00640 1 -- 172.21.15.131:0/2352006874 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a7001b070 con 0x7f7a6c004800 2024-04-28T12:38:23.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.012+0000 7f7a7fe00640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7a6406c8f0 0x7f7a6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.012+0000 7f7a7fe00640 1 -- 172.21.15.131:0/2352006874 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7a7008c150 con 0x7f7a6c004800 2024-04-28T12:38:23.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.012+0000 7f7a87400640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7a6406c8f0 0x7f7a6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.014+0000 7f7a87400640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7a6406c8f0 0x7f7a6406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7a78035000 tx=0x7f7a7802f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:23.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.016+0000 7f7a7fe00640 1 -- 172.21.15.131:0/2352006874 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7a7005a180 con 0x7f7a6c004800 2024-04-28T12:38:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.224+0000 7f7a8ec00640 1 -- 172.21.15.131:0/2352006874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7a880b1850 con 0x7f7a6c004800 2024-04-28T12:38:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.225+0000 7f7a7fe00640 1 -- 172.21.15.131:0/2352006874 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f7a70006c30 con 0x7f7a6c004800 2024-04-28T12:38:23.221 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:23.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.227+0000 7f7a8ec00640 1 -- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7a6406c8f0 msgr2=0x7f7a6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.227+0000 7f7a8ec00640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7a6406c8f0 0x7f7a6406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7a78035000 tx=0x7f7a7802f000 comp rx=0 tx=0).stop 2024-04-28T12:38:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.227+0000 7f7a8ec00640 1 -- 172.21.15.131:0/2352006874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c004800 msgr2=0x7f7a880b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.227+0000 7f7a8ec00640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c004800 0x7f7a880b1a70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000f650 tx=0x7f7a7000fb10 comp rx=0 tx=0).stop 2024-04-28T12:38:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.228+0000 7f7a8ec00640 1 -- 172.21.15.131:0/2352006874 shutdown_connections 2024-04-28T12:38:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.228+0000 7f7a8ec00640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7a6406c8f0 0x7f7a6406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.228+0000 7f7a8ec00640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a8815c770 0x7f7a880afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.228+0000 7f7a8ec00640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a8814dc50 0x7f7a880af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.228+0000 7f7a8ec00640 1 --2- 172.21.15.131:0/2352006874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c004800 0x7f7a880b1a70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.228+0000 7f7a8ec00640 1 -- 172.21.15.131:0/2352006874 >> 172.21.15.131:0/2352006874 conn(0x7f7a880b38a0 msgr2=0x7f7a88072e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.228+0000 7f7a8ec00640 1 -- 172.21.15.131:0/2352006874 shutdown_connections 2024-04-28T12:38:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.228+0000 7f7a8ec00640 1 -- 172.21.15.131:0/2352006874 wait complete. 2024-04-28T12:38:23.244 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:23.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.438+0000 7fcb6f600640 1 Processor -- start 2024-04-28T12:38:23.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.438+0000 7fcb6f600640 1 -- start start 2024-04-28T12:38:23.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.440+0000 7fcb6f600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcb68071eb0 0x7fcb68072290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.440+0000 7fcb6f600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb680727d0 con 0x7fcb68163bd0 2024-04-28T12:38:23.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.440+0000 7fcb6f600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb680b4570 con 0x7fcb6806df10 2024-04-28T12:38:23.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.440+0000 7fcb6f600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcb680b5180 con 0x7fcb68071eb0 2024-04-28T12:38:23.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.440+0000 7fcb67e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcb68163bd0 0x7fcb680b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42990/0 (socket says 172.21.15.131:42990) 2024-04-28T12:38:23.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.440+0000 7fcb67e00640 1 -- 172.21.15.131:0/620665592 learned_addr learned my addr 172.21.15.131:0/620665592 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:23.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb6cc00640 1 --2- 172.21.15.131:0/620665592 >> v2:172.21.15.157:3300/0 conn(0x7fcb68071eb0 0x7fcb68072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3146560665 0 0) 0x7fcb680727d0 con 0x7fcb68163bd0 2024-04-28T12:38:23.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb50003520 con 0x7fcb68163bd0 2024-04-28T12:38:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 352546885 0 0) 0x7fcb50003520 con 0x7fcb68163bd0 2024-04-28T12:38:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 >> v2:172.21.15.157:3300/0 conn(0x7fcb68071eb0 msgr2=0x7fcb68072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 --2- 172.21.15.131:0/620665592 >> v2:172.21.15.157:3300/0 conn(0x7fcb68071eb0 0x7fcb68072290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 >> v1:172.21.15.157:6789/0 conn(0x7fcb6806df10 legacy=0x7fcb6806e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb680b4850 con 0x7fcb68163bd0 2024-04-28T12:38:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fcb54001670 con 0x7fcb68163bd0 2024-04-28T12:38:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 >> v1:172.21.15.131:6789/0 conn(0x7fcb68163bd0 legacy=0x7fcb680b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 --2- 172.21.15.131:0/620665592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb500039b0 0x7fcb50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 --2- 172.21.15.131:0/620665592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 --2- 172.21.15.131:0/620665592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb500051c0 0x7fcb5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb680b5180 con 0x7fcb50004800 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5000c160 con 0x7fcb500051c0 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.441+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5000c360 con 0x7fcb500039b0 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.442+0000 7fcb67e00640 1 --2- 172.21.15.131:0/620665592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.442+0000 7fcb6cc00640 1 --2- 172.21.15.131:0/620665592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb500039b0 0x7fcb50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.442+0000 7fcb67e00640 1 -- 172.21.15.131:0/620665592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb500039b0 msgr2=0x7fcb50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.442+0000 7fcb67e00640 1 --2- 172.21.15.131:0/620665592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb500039b0 0x7fcb50003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.442+0000 7fcb67e00640 1 -- 172.21.15.131:0/620665592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb500051c0 msgr2=0x7fcb5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.442+0000 7fcb6d600640 1 --2- 172.21.15.131:0/620665592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb500051c0 0x7fcb5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.442+0000 7fcb67e00640 1 --2- 172.21.15.131:0/620665592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb500051c0 0x7fcb5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.442+0000 7fcb67e00640 1 -- 172.21.15.131:0/620665592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb50003520 con 0x7fcb50004800 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.442+0000 7fcb6d600640 1 --2- 172.21.15.131:0/620665592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb500051c0 0x7fcb5000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.442+0000 7fcb6d600640 1 -- 172.21.15.131:0/620665592 reap_dead start 2024-04-28T12:38:23.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.443+0000 7fcb67e00640 1 --2- 172.21.15.131:0/620665592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c210 tx=0x7fcb50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbd9da1347cab2d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:23.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.443+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54007d30 con 0x7fcb50004800 2024-04-28T12:38:23.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.443+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb54005ad0 con 0x7fcb50004800 2024-04-28T12:38:23.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.443+0000 7fcb67400640 1 -- 172.21.15.131:0/620665592 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54007070 con 0x7fcb50004800 2024-04-28T12:38:23.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.444+0000 7fcb6f600640 1 -- 172.21.15.131:0/620665592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb50004800 msgr2=0x7fcb50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.444+0000 7fcb6f600640 1 --2- 172.21.15.131:0/620665592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c210 tx=0x7fcb50005830 comp rx=0 tx=0).stop 2024-04-28T12:38:23.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.444+0000 7fcb6f600640 1 -- 172.21.15.131:0/620665592 shutdown_connections 2024-04-28T12:38:23.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.444+0000 7fcb6f600640 1 --2- 172.21.15.131:0/620665592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.444+0000 7fcb6f600640 1 -- 172.21.15.131:0/620665592 >> 172.21.15.131:0/620665592 conn(0x7fcb680b2900 msgr2=0x7fcb680b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:23.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.444+0000 7fcb6f600640 1 -- 172.21.15.131:0/620665592 shutdown_connections 2024-04-28T12:38:23.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.444+0000 7fcb6f600640 1 -- 172.21.15.131:0/620665592 wait complete. 2024-04-28T12:38:23.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.446+0000 7fcb6f600640 1 Processor -- start 2024-04-28T12:38:23.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.446+0000 7fcb6f600640 1 -- start start 2024-04-28T12:38:23.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.447+0000 7fcb6f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb50004800 0x7fcb6815eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.447+0000 7fcb6f600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb6815cdc0 0x7fcb6815d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.447+0000 7fcb6f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb68159cb0 0x7fcb6815a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.447+0000 7fcb6f600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb680d0620 con 0x7fcb6815cdc0 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.447+0000 7fcb6f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb680b4e10 con 0x7fcb68159cb0 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.447+0000 7fcb6f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcb68072290 con 0x7fcb50004800 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.447+0000 7fcb67e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb6815cdc0 0x7fcb6815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.447+0000 7fcb67e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb6815cdc0 0x7fcb6815d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54094/0 (socket says 172.21.15.131:54094) 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.447+0000 7fcb67e00640 1 -- 172.21.15.131:0/2233504389 learned_addr learned my addr 172.21.15.131:0/2233504389 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.447+0000 7fcb6cc00640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb50004800 0x7fcb6815eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.447+0000 7fcb6d600640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb68159cb0 0x7fcb6815a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.448+0000 7fcb67e00640 1 -- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb50004800 msgr2=0x7fcb6815eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.448+0000 7fcb67e00640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb50004800 0x7fcb6815eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.448+0000 7fcb67e00640 1 -- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb68159cb0 msgr2=0x7fcb6815a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.448+0000 7fcb67e00640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb68159cb0 0x7fcb6815a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.448+0000 7fcb67e00640 1 -- 172.21.15.131:0/2233504389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6815f410 con 0x7fcb6815cdc0 2024-04-28T12:38:23.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.448+0000 7fcb6cc00640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb50004800 0x7fcb6815eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:38:23.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.448+0000 7fcb67e00640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb6815cdc0 0x7fcb6815d240 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c330 tx=0x7fcb54001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:23.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.448+0000 7fcb64c00640 1 -- 172.21.15.131:0/2233504389 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54041070 con 0x7fcb6815cdc0 2024-04-28T12:38:23.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.448+0000 7fcb64c00640 1 -- 172.21.15.131:0/2233504389 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb54000ed0 con 0x7fcb6815cdc0 2024-04-28T12:38:23.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.448+0000 7fcb6f600640 1 -- 172.21.15.131:0/2233504389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb68158c10 con 0x7fcb6815cdc0 2024-04-28T12:38:23.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.448+0000 7fcb64c00640 1 -- 172.21.15.131:0/2233504389 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54007290 con 0x7fcb6815cdc0 2024-04-28T12:38:23.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.448+0000 7fcb6f600640 1 -- 172.21.15.131:0/2233504389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb681591d0 con 0x7fcb6815cdc0 2024-04-28T12:38:23.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.449+0000 7fcb6f600640 1 -- 172.21.15.131:0/2233504389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb68069350 con 0x7fcb6815cdc0 2024-04-28T12:38:23.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.450+0000 7fcb64c00640 1 -- 172.21.15.131:0/2233504389 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb54044070 con 0x7fcb6815cdc0 2024-04-28T12:38:23.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.451+0000 7fcb64c00640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcb4806c8f0 0x7fcb4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.451+0000 7fcb64c00640 1 -- 172.21.15.131:0/2233504389 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcb540ac650 con 0x7fcb6815cdc0 2024-04-28T12:38:23.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.452+0000 7fcb6cc00640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcb4806c8f0 0x7fcb4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.453+0000 7fcb6cc00640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcb4806c8f0 0x7fcb4806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009b50 tx=0x7fcb5c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:23.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.456+0000 7fcb64c00640 1 -- 172.21.15.131:0/2233504389 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb5407a680 con 0x7fcb6815cdc0 2024-04-28T12:38:23.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.652+0000 7fcb6f600640 1 -- 172.21.15.131:0/2233504389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb680d1b40 con 0x7fcb6815cdc0 2024-04-28T12:38:23.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.653+0000 7fcb64c00640 1 -- 172.21.15.131:0/2233504389 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fcb54079f80 con 0x7fcb6815cdc0 2024-04-28T12:38:23.650 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:23.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.656+0000 7fcb6f600640 1 -- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcb4806c8f0 msgr2=0x7fcb4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.656+0000 7fcb6f600640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcb4806c8f0 0x7fcb4806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009b50 tx=0x7fcb5c008040 comp rx=0 tx=0).stop 2024-04-28T12:38:23.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.656+0000 7fcb6f600640 1 -- 172.21.15.131:0/2233504389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb6815cdc0 msgr2=0x7fcb6815d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.656+0000 7fcb6f600640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb6815cdc0 0x7fcb6815d240 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c330 tx=0x7fcb54001a40 comp rx=0 tx=0).stop 2024-04-28T12:38:23.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.656+0000 7fcb6f600640 1 -- 172.21.15.131:0/2233504389 shutdown_connections 2024-04-28T12:38:23.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.656+0000 7fcb6f600640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcb4806c8f0 0x7fcb4806edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.656+0000 7fcb6f600640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb68159cb0 0x7fcb6815a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.656+0000 7fcb6f600640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb6815cdc0 0x7fcb6815d240 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.656+0000 7fcb6f600640 1 --2- 172.21.15.131:0/2233504389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb50004800 0x7fcb6815eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.656+0000 7fcb6f600640 1 -- 172.21.15.131:0/2233504389 >> 172.21.15.131:0/2233504389 conn(0x7fcb680b2900 msgr2=0x7fcb68163140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:23.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.656+0000 7fcb6f600640 1 -- 172.21.15.131:0/2233504389 shutdown_connections 2024-04-28T12:38:23.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.656+0000 7fcb6f600640 1 -- 172.21.15.131:0/2233504389 wait complete. 2024-04-28T12:38:23.671 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:23.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.875+0000 7fd415800640 1 Processor -- start 2024-04-28T12:38:23.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.876+0000 7fd415800640 1 -- start start 2024-04-28T12:38:23.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.878+0000 7fd415800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd41006f3a0 0x7fd4100b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.878+0000 7fd415800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd41015c2a0 con 0x7fd41015bd60 2024-04-28T12:38:23.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.878+0000 7fd415800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd41015c580 con 0x7fd41015b980 2024-04-28T12:38:23.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.878+0000 7fd415800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd41015d060 con 0x7fd41006f3a0 2024-04-28T12:38:23.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.878+0000 7fd407e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd41006f3a0 0x7fd4100b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.878+0000 7fd407e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd41006f3a0 0x7fd4100b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54100/0 (socket says 172.21.15.131:54100) 2024-04-28T12:38:23.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.878+0000 7fd407e00640 1 -- 172.21.15.131:0/3064366207 learned_addr learned my addr 172.21.15.131:0/3064366207 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:23.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.878+0000 7fd414e00640 1 --1- 172.21.15.131:0/3064366207 >> v1:172.21.15.131:6789/0 conn(0x7fd41015bd60 0x7fd41015f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42994/0 (socket says 172.21.15.131:42994) 2024-04-28T12:38:23.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.878+0000 7fd407e00640 1 -- 172.21.15.131:0/3064366207 >> v1:172.21.15.157:6789/0 conn(0x7fd41015b980 legacy=0x7fd41006ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:23.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.878+0000 7fd407e00640 1 -- 172.21.15.131:0/3064366207 >> v1:172.21.15.131:6789/0 conn(0x7fd41015bd60 legacy=0x7fd41015f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.878+0000 7fd407e00640 1 -- 172.21.15.131:0/3064366207 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41015c860 con 0x7fd41006f3a0 2024-04-28T12:38:23.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.879+0000 7fd407e00640 1 --2- 172.21.15.131:0/3064366207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41006f3a0 0x7fd4100b4610 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009870 tx=0x7fd3f802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184081bf050a98bd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:23.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.880+0000 7fd40f400640 1 -- 172.21.15.131:0/3064366207 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8037070 con 0x7fd41006f3a0 2024-04-28T12:38:23.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.880+0000 7fd40f400640 1 -- 172.21.15.131:0/3064366207 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f8032440 con 0x7fd41006f3a0 2024-04-28T12:38:23.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.880+0000 7fd40f400640 1 -- 172.21.15.131:0/3064366207 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8032810 con 0x7fd41006f3a0 2024-04-28T12:38:23.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.880+0000 7fd415800640 1 -- 172.21.15.131:0/3064366207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41006f3a0 msgr2=0x7fd4100b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.880+0000 7fd415800640 1 --2- 172.21.15.131:0/3064366207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41006f3a0 0x7fd4100b4610 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009870 tx=0x7fd3f802a2a0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.880+0000 7fd415800640 1 -- 172.21.15.131:0/3064366207 shutdown_connections 2024-04-28T12:38:23.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.880+0000 7fd415800640 1 --2- 172.21.15.131:0/3064366207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41006f3a0 0x7fd4100b4610 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.880+0000 7fd415800640 1 -- 172.21.15.131:0/3064366207 >> 172.21.15.131:0/3064366207 conn(0x7fd410068090 msgr2=0x7fd4100684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:23.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.881+0000 7fd415800640 1 -- 172.21.15.131:0/3064366207 shutdown_connections 2024-04-28T12:38:23.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.881+0000 7fd415800640 1 -- 172.21.15.131:0/3064366207 wait complete. 2024-04-28T12:38:23.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.882+0000 7fd415800640 1 Processor -- start 2024-04-28T12:38:23.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.882+0000 7fd415800640 1 -- start start 2024-04-28T12:38:23.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.883+0000 7fd415800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd41006f3a0 0x7fd4101471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.883+0000 7fd415800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41015b980 0x7fd410154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.883+0000 7fd415800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd41015bd60 0x7fd410155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.883+0000 7fd415800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd4100b4b50 con 0x7fd41015b980 2024-04-28T12:38:23.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.883+0000 7fd415800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd4100b52b0 con 0x7fd41015bd60 2024-04-28T12:38:23.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.883+0000 7fd415800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd4100b4cd0 con 0x7fd41006f3a0 2024-04-28T12:38:23.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.884+0000 7fd407e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41015b980 0x7fd410154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.884+0000 7fd407e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41015b980 0x7fd410154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54112/0 (socket says 172.21.15.131:54112) 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.884+0000 7fd407e00640 1 -- 172.21.15.131:0/1401439764 learned_addr learned my addr 172.21.15.131:0/1401439764 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.884+0000 7fd407e00640 1 -- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd41006f3a0 msgr2=0x7fd4101471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.884+0000 7fd407e00640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd41006f3a0 0x7fd4101471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.884+0000 7fd414e00640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd41015bd60 0x7fd410155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.884+0000 7fd407e00640 1 -- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd41015bd60 msgr2=0x7fd410155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.884+0000 7fd407e00640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd41015bd60 0x7fd410155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.884+0000 7fd407e00640 1 -- 172.21.15.131:0/1401439764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd410154510 con 0x7fd41015b980 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.884+0000 7fd407e00640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41015b980 0x7fd410154ce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009870 tx=0x7fd3f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.885+0000 7fd40cc00640 1 -- 172.21.15.131:0/1401439764 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8041020 con 0x7fd41015b980 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.885+0000 7fd40cc00640 1 -- 172.21.15.131:0/1401439764 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f8030970 con 0x7fd41015b980 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.885+0000 7fd40cc00640 1 -- 172.21.15.131:0/1401439764 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8037040 con 0x7fd41015b980 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.885+0000 7fd415800640 1 -- 172.21.15.131:0/1401439764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4101e4c40 con 0x7fd41015b980 2024-04-28T12:38:23.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.885+0000 7fd415800640 1 -- 172.21.15.131:0/1401439764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4101e5200 con 0x7fd41015b980 2024-04-28T12:38:23.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.886+0000 7fd415800640 1 -- 172.21.15.131:0/1401439764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4100506c0 con 0x7fd41015b980 2024-04-28T12:38:23.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.887+0000 7fd40cc00640 1 -- 172.21.15.131:0/1401439764 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3f8044020 con 0x7fd41015b980 2024-04-28T12:38:23.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.888+0000 7fd40cc00640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd3ec06c920 0x7fd3ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:23.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.888+0000 7fd40cc00640 1 -- 172.21.15.131:0/1401439764 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd3f80ac690 con 0x7fd41015b980 2024-04-28T12:38:23.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.888+0000 7fd40fe00640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd3ec06c920 0x7fd3ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:23.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.890+0000 7fd40fe00640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd3ec06c920 0x7fd3ec06ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd400003cc0 tx=0x7fd4000079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:23.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:23.893+0000 7fd40cc00640 1 -- 172.21.15.131:0/1401439764 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3f804b070 con 0x7fd41015b980 2024-04-28T12:38:24.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.078+0000 7fd415800640 1 -- 172.21.15.131:0/1401439764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd4100b3ee0 con 0x7fd41015b980 2024-04-28T12:38:24.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.079+0000 7fd40cc00640 1 -- 172.21.15.131:0/1401439764 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd3f807a620 con 0x7fd41015b980 2024-04-28T12:38:24.076 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:24.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.081+0000 7fd415800640 1 -- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd3ec06c920 msgr2=0x7fd3ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.081+0000 7fd415800640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd3ec06c920 0x7fd3ec06ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd400003cc0 tx=0x7fd4000079b0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.081+0000 7fd415800640 1 -- 172.21.15.131:0/1401439764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41015b980 msgr2=0x7fd410154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.081+0000 7fd415800640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41015b980 0x7fd410154ce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009870 tx=0x7fd3f80046c0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.082+0000 7fd415800640 1 -- 172.21.15.131:0/1401439764 shutdown_connections 2024-04-28T12:38:24.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.082+0000 7fd415800640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd3ec06c920 0x7fd3ec06ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.082+0000 7fd415800640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd41015bd60 0x7fd410155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.082+0000 7fd415800640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd41015b980 0x7fd410154ce0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.082+0000 7fd415800640 1 --2- 172.21.15.131:0/1401439764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd41006f3a0 0x7fd4101471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.082+0000 7fd415800640 1 -- 172.21.15.131:0/1401439764 >> 172.21.15.131:0/1401439764 conn(0x7fd410068090 msgr2=0x7fd410156ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:24.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.083+0000 7fd415800640 1 -- 172.21.15.131:0/1401439764 shutdown_connections 2024-04-28T12:38:24.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.083+0000 7fd415800640 1 -- 172.21.15.131:0/1401439764 wait complete. 2024-04-28T12:38:24.098 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.295+0000 7f1b31800640 1 Processor -- start 2024-04-28T12:38:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.296+0000 7f1b31800640 1 -- start start 2024-04-28T12:38:24.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.297+0000 7f1b31800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1b2c0d1620 0x7f1b2c0d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:24.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.297+0000 7f1b31800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b2c0d1a00 0x7f1b2c0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:24.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.297+0000 7f1b31800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b2c0d0480 con 0x7f1b2c0b50b0 2024-04-28T12:38:24.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.297+0000 7f1b31800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1b2c0d0ce0 con 0x7f1b2c0d1620 2024-04-28T12:38:24.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.297+0000 7f1b31800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c0d0720 con 0x7f1b2c0d1a00 2024-04-28T12:38:24.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.297+0000 7f1b2be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1b2c0d1620 0x7f1b2c0d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:24.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.297+0000 7f1b2be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1b2c0d1620 0x7f1b2c0d5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54122/0 (socket says 172.21.15.131:54122) 2024-04-28T12:38:24.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.297+0000 7f1b2be00640 1 -- 172.21.15.131:0/662442899 learned_addr learned my addr 172.21.15.131:0/662442899 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:24.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.297+0000 7f1b30e00640 1 --1- 172.21.15.131:0/662442899 >> v1:172.21.15.131:6789/0 conn(0x7f1b2c0b50b0 0x7f1b2c0b5540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43002/0 (socket says 172.21.15.131:43002) 2024-04-28T12:38:24.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.298+0000 7f1b2b400640 1 --2- 172.21.15.131:0/662442899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b2c0d1a00 0x7f1b2c0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:24.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.298+0000 7f1b2be00640 1 -- 172.21.15.131:0/662442899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b2c0d1a00 msgr2=0x7f1b2c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:24.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.298+0000 7f1b2be00640 1 --2- 172.21.15.131:0/662442899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b2c0d1a00 0x7f1b2c0b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.298+0000 7f1b2be00640 1 -- 172.21.15.131:0/662442899 >> v1:172.21.15.131:6789/0 conn(0x7f1b2c0b50b0 legacy=0x7f1b2c0b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.298+0000 7f1b2be00640 1 -- 172.21.15.131:0/662442899 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c0d0a00 con 0x7f1b2c0d1620 2024-04-28T12:38:24.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.299+0000 7f1b2be00640 1 --2- 172.21.15.131:0/662442899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c0d1620 0x7f1b2c0d5750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009e50 tx=0x7f1b1c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da9445590e7de11 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:24.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.299+0000 7f1b2aa00640 1 -- 172.21.15.131:0/662442899 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c036070 con 0x7f1b2c0d1620 2024-04-28T12:38:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.299+0000 7f1b2aa00640 1 -- 172.21.15.131:0/662442899 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1c031600 con 0x7f1b2c0d1620 2024-04-28T12:38:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.299+0000 7f1b2aa00640 1 -- 172.21.15.131:0/662442899 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0319d0 con 0x7f1b2c0d1620 2024-04-28T12:38:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.300+0000 7f1b31800640 1 -- 172.21.15.131:0/662442899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c0d1620 msgr2=0x7f1b2c0d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.300+0000 7f1b31800640 1 --2- 172.21.15.131:0/662442899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c0d1620 0x7f1b2c0d5750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009e50 tx=0x7f1b1c02a680 comp rx=0 tx=0).stop 2024-04-28T12:38:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.300+0000 7f1b31800640 1 -- 172.21.15.131:0/662442899 shutdown_connections 2024-04-28T12:38:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.300+0000 7f1b31800640 1 --2- 172.21.15.131:0/662442899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b2c0d1a00 0x7f1b2c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.300+0000 7f1b31800640 1 --2- 172.21.15.131:0/662442899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c0d1620 0x7f1b2c0d5750 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.300+0000 7f1b31800640 1 -- 172.21.15.131:0/662442899 >> 172.21.15.131:0/662442899 conn(0x7f1b2c0d4c30 msgr2=0x7f1b2c0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.300+0000 7f1b31800640 1 -- 172.21.15.131:0/662442899 shutdown_connections 2024-04-28T12:38:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.300+0000 7f1b31800640 1 -- 172.21.15.131:0/662442899 wait complete. 2024-04-28T12:38:24.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.302+0000 7f1b31800640 1 Processor -- start 2024-04-28T12:38:24.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.302+0000 7f1b31800640 1 -- start start 2024-04-28T12:38:24.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.302+0000 7f1b31800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2c0b50b0 0x7f1b2c14b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:24.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.302+0000 7f1b31800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c0d1620 0x7f1b2c0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b31800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b2c0d1a00 0x7f1b2c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b31800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c0b5c50 con 0x7f1b2c0d1620 2024-04-28T12:38:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b31800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c0d1e60 con 0x7f1b2c0b50b0 2024-04-28T12:38:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b31800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c0b5dd0 con 0x7f1b2c0d1a00 2024-04-28T12:38:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b2b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c0d1620 0x7f1b2c0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b2b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c0d1620 0x7f1b2c0cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54134/0 (socket says 172.21.15.131:54134) 2024-04-28T12:38:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b2b400640 1 -- 172.21.15.131:0/2961597994 learned_addr learned my addr 172.21.15.131:0/2961597994 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b2b400640 1 -- 172.21.15.131:0/2961597994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b2c0d1a00 msgr2=0x7f1b2c06d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b2b400640 1 --2- 172.21.15.131:0/2961597994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b2c0d1a00 0x7f1b2c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b2b400640 1 -- 172.21.15.131:0/2961597994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2c0b50b0 msgr2=0x7f1b2c14b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b2b400640 1 --2- 172.21.15.131:0/2961597994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2c0b50b0 0x7f1b2c14b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b2b400640 1 -- 172.21.15.131:0/2961597994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c0cc750 con 0x7f1b2c0d1620 2024-04-28T12:38:24.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.303+0000 7f1b2b400640 1 --2- 172.21.15.131:0/2961597994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c0d1620 0x7f1b2c0cbd00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400c7f0 tx=0x7f1b1400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:24.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.304+0000 7f1b23e00640 1 -- 172.21.15.131:0/2961597994 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14015070 con 0x7f1b2c0d1620 2024-04-28T12:38:24.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.304+0000 7f1b23e00640 1 -- 172.21.15.131:0/2961597994 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b14004220 con 0x7f1b2c0d1620 2024-04-28T12:38:24.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.304+0000 7f1b23e00640 1 -- 172.21.15.131:0/2961597994 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14007620 con 0x7f1b2c0d1620 2024-04-28T12:38:24.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.304+0000 7f1b31800640 1 -- 172.21.15.131:0/2961597994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2c0cca30 con 0x7f1b2c0d1620 2024-04-28T12:38:24.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.304+0000 7f1b31800640 1 -- 172.21.15.131:0/2961597994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2c0ccff0 con 0x7f1b2c0d1620 2024-04-28T12:38:24.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.305+0000 7f1b31800640 1 -- 172.21.15.131:0/2961597994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2c069350 con 0x7f1b2c0d1620 2024-04-28T12:38:24.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.306+0000 7f1b23e00640 1 -- 172.21.15.131:0/2961597994 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b140079d0 con 0x7f1b2c0d1620 2024-04-28T12:38:24.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.307+0000 7f1b23e00640 1 --2- 172.21.15.131:0/2961597994 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1b0806c920 0x7f1b0806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:24.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.307+0000 7f1b23e00640 1 -- 172.21.15.131:0/2961597994 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1b1408bf60 con 0x7f1b2c0d1620 2024-04-28T12:38:24.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.308+0000 7f1b2be00640 1 --2- 172.21.15.131:0/2961597994 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1b0806c920 0x7f1b0806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:24.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.309+0000 7f1b2be00640 1 --2- 172.21.15.131:0/2961597994 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1b0806c920 0x7f1b0806ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c033000 tx=0x7f1b1c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:24.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.312+0000 7f1b23e00640 1 -- 172.21.15.131:0/2961597994 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b14059f80 con 0x7f1b2c0d1620 2024-04-28T12:38:24.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.510+0000 7f1b31800640 1 -- 172.21.15.131:0/2961597994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b2c0b1860 con 0x7f1b2c0d1620 2024-04-28T12:38:24.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.513+0000 7f1b23e00640 1 -- 172.21.15.131:0/2961597994 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7f1b14059880 con 0x7f1b2c0d1620 2024-04-28T12:38:24.510 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:24.510 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.515+0000 7f1b31800640 1 -- 172.21.15.131:0/2961597994 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1b0806c920 msgr2=0x7f1b0806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.515+0000 7f1b31800640 1 --2- 172.21.15.131:0/2961597994 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1b0806c920 0x7f1b0806ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c033000 tx=0x7f1b1c02e000 comp rx=0 tx=0).stop 2024-04-28T12:38:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.515+0000 7f1b31800640 1 -- 172.21.15.131:0/2961597994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c0d1620 msgr2=0x7f1b2c0cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.515+0000 7f1b31800640 1 --2- 172.21.15.131:0/2961597994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c0d1620 0x7f1b2c0cbd00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400c7f0 tx=0x7f1b1400ccb0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.515+0000 7f1b31800640 1 -- 172.21.15.131:0/2961597994 shutdown_connections 2024-04-28T12:38:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.515+0000 7f1b31800640 1 --2- 172.21.15.131:0/2961597994 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1b0806c920 0x7f1b0806ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.515+0000 7f1b31800640 1 --2- 172.21.15.131:0/2961597994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b2c0d1a00 0x7f1b2c06d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.515+0000 7f1b31800640 1 --2- 172.21.15.131:0/2961597994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2c0d1620 0x7f1b2c0cbd00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.515+0000 7f1b31800640 1 --2- 172.21.15.131:0/2961597994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b2c0b50b0 0x7f1b2c14b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.515+0000 7f1b31800640 1 -- 172.21.15.131:0/2961597994 >> 172.21.15.131:0/2961597994 conn(0x7f1b2c0d4c30 msgr2=0x7f1b2c0d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.516+0000 7f1b31800640 1 -- 172.21.15.131:0/2961597994 shutdown_connections 2024-04-28T12:38:24.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.516+0000 7f1b31800640 1 -- 172.21.15.131:0/2961597994 wait complete. 2024-04-28T12:38:24.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:24.529 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:24.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.734+0000 7f85a6600640 1 Processor -- start 2024-04-28T12:38:24.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.735+0000 7f85a6600640 1 -- start start 2024-04-28T12:38:24.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.736+0000 7f85a6600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f85a00d06e0 0x7f85a006ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:24.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.736+0000 7f85a6600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85a00b54f0 con 0x7f85a0145ba0 2024-04-28T12:38:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.736+0000 7f85a6600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85a006df40 con 0x7f85a006f240 2024-04-28T12:38:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.736+0000 7f85a6600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f85a006e220 con 0x7f85a00d06e0 2024-04-28T12:38:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.736+0000 7f859f400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f85a0145ba0 0x7f85a015d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43016/0 (socket says 172.21.15.131:43016) 2024-04-28T12:38:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.737+0000 7f859f400640 1 -- 172.21.15.131:0/315256620 learned_addr learned my addr 172.21.15.131:0/315256620 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.737+0000 7f8597e00640 1 --2- 172.21.15.131:0/315256620 >> v2:172.21.15.157:3300/0 conn(0x7f85a00d06e0 0x7f85a006ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.737+0000 7f859ea00640 1 -- 172.21.15.131:0/315256620 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3853187790 0 0) 0x7f85a00b54f0 con 0x7f85a0145ba0 2024-04-28T12:38:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.737+0000 7f859ea00640 1 -- 172.21.15.131:0/315256620 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8584003520 con 0x7f85a0145ba0 2024-04-28T12:38:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.738+0000 7f8597e00640 1 -- 172.21.15.131:0/315256620 >> v1:172.21.15.157:6789/0 conn(0x7f85a006f240 legacy=0x7f85a00b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.738+0000 7f8597e00640 1 -- 172.21.15.131:0/315256620 >> v1:172.21.15.131:6789/0 conn(0x7f85a0145ba0 legacy=0x7f85a015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.738+0000 7f859ea00640 1 -- 172.21.15.131:0/315256620 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4264994536 0 0) 0x7f8584003520 con 0x7f85a0145ba0 2024-04-28T12:38:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.738+0000 7f8597e00640 1 -- 172.21.15.131:0/315256620 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a006e500 con 0x7f85a00d06e0 2024-04-28T12:38:24.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.739+0000 7f8597e00640 1 --2- 172.21.15.131:0/315256620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a00d06e0 0x7f85a006ed00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85880097a0 tx=0x7f858802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503d2593fbe8f1af server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:24.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.739+0000 7f859ea00640 1 -- 172.21.15.131:0/315256620 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588037070 con 0x7f85a00d06e0 2024-04-28T12:38:24.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.739+0000 7f859ea00640 1 -- 172.21.15.131:0/315256620 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f858802ac50 con 0x7f85a00d06e0 2024-04-28T12:38:24.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.739+0000 7f859ea00640 1 -- 172.21.15.131:0/315256620 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590001670 con 0x7f85a00d06e0 2024-04-28T12:38:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.740+0000 7f85a6600640 1 -- 172.21.15.131:0/315256620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a00d06e0 msgr2=0x7f85a006ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.740+0000 7f85a6600640 1 --2- 172.21.15.131:0/315256620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a00d06e0 0x7f85a006ed00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85880097a0 tx=0x7f858802a1c0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.740+0000 7f85a6600640 1 -- 172.21.15.131:0/315256620 shutdown_connections 2024-04-28T12:38:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.740+0000 7f85a6600640 1 --2- 172.21.15.131:0/315256620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a00d06e0 0x7f85a006ed00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.740+0000 7f85a6600640 1 -- 172.21.15.131:0/315256620 >> 172.21.15.131:0/315256620 conn(0x7f85a00b2900 msgr2=0x7f85a00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.740+0000 7f85a6600640 1 -- 172.21.15.131:0/315256620 shutdown_connections 2024-04-28T12:38:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.740+0000 7f85a6600640 1 -- 172.21.15.131:0/315256620 wait complete. 2024-04-28T12:38:24.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.742+0000 7f85a6600640 1 Processor -- start 2024-04-28T12:38:24.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.742+0000 7f85a6600640 1 -- start start 2024-04-28T12:38:24.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.743+0000 7f85a6600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a006f240 0x7f85a015d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:24.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.743+0000 7f85a6600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85a00d06e0 0x7f85a013ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:24.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.743+0000 7f85a6600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85a0145ba0 0x7f85a013f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:24.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.743+0000 7f85a6600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0145f80 con 0x7f85a00d06e0 2024-04-28T12:38:24.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.743+0000 7f85a6600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85a006f670 con 0x7f85a006f240 2024-04-28T12:38:24.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.743+0000 7f85a6600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f85a015df90 con 0x7f85a0145ba0 2024-04-28T12:38:24.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.743+0000 7f8597e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85a00d06e0 0x7f85a013ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:24.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.743+0000 7f8597e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85a00d06e0 0x7f85a013ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54140/0 (socket says 172.21.15.131:54140) 2024-04-28T12:38:24.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.743+0000 7f8597e00640 1 -- 172.21.15.131:0/2827580118 learned_addr learned my addr 172.21.15.131:0/2827580118 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:24.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f859fe00640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85a0145ba0 0x7f85a013f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:24.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f8597e00640 1 -- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85a0145ba0 msgr2=0x7f85a013f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f859f400640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a006f240 0x7f85a015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f8597e00640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85a0145ba0 0x7f85a013f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f8597e00640 1 -- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a006f240 msgr2=0x7f85a015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f8597e00640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a006f240 0x7f85a015d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f8597e00640 1 -- 172.21.15.131:0/2827580118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a015e270 con 0x7f85a00d06e0 2024-04-28T12:38:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f859f400640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a006f240 0x7f85a015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f8597e00640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85a00d06e0 0x7f85a013ec00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8588035040 tx=0x7f85880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f8597400640 1 -- 172.21.15.131:0/2827580118 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588041020 con 0x7f85a00d06e0 2024-04-28T12:38:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f8597400640 1 -- 172.21.15.131:0/2827580118 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85880044c0 con 0x7f85a00d06e0 2024-04-28T12:38:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f8597400640 1 -- 172.21.15.131:0/2827580118 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588037040 con 0x7f85a00d06e0 2024-04-28T12:38:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f85a6600640 1 -- 172.21.15.131:0/2827580118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a015e550 con 0x7f85a00d06e0 2024-04-28T12:38:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.744+0000 7f85a6600640 1 -- 172.21.15.131:0/2827580118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a015eb10 con 0x7f85a00d06e0 2024-04-28T12:38:24.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.746+0000 7f85a6600640 1 -- 172.21.15.131:0/2827580118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a0069350 con 0x7f85a00d06e0 2024-04-28T12:38:24.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.746+0000 7f8597400640 1 -- 172.21.15.131:0/2827580118 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8588044020 con 0x7f85a00d06e0 2024-04-28T12:38:24.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.747+0000 7f8597400640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f857c06c8f0 0x7f857c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:24.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.747+0000 7f8597400640 1 -- 172.21.15.131:0/2827580118 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f85880ac3f0 con 0x7f85a00d06e0 2024-04-28T12:38:24.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.748+0000 7f859f400640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f857c06c8f0 0x7f857c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:24.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.749+0000 7f859f400640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f857c06c8f0 0x7f857c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8590001eb0 tx=0x7f8590009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:24.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.752+0000 7f8597400640 1 -- 172.21.15.131:0/2827580118 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f858804b030 con 0x7f85a00d06e0 2024-04-28T12:38:24.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.957+0000 7f85a6600640 1 -- 172.21.15.131:0/2827580118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f85a00cf610 con 0x7f85a00d06e0 2024-04-28T12:38:24.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.958+0000 7f8597400640 1 -- 172.21.15.131:0/2827580118 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f85880313a0 con 0x7f85a00d06e0 2024-04-28T12:38:24.954 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.960+0000 7f85a6600640 1 -- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f857c06c8f0 msgr2=0x7f857c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.960+0000 7f85a6600640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f857c06c8f0 0x7f857c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8590001eb0 tx=0x7f8590009040 comp rx=0 tx=0).stop 2024-04-28T12:38:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.960+0000 7f85a6600640 1 -- 172.21.15.131:0/2827580118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85a00d06e0 msgr2=0x7f85a013ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.960+0000 7f85a6600640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85a00d06e0 0x7f85a013ec00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8588035040 tx=0x7f85880046c0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.961+0000 7f85a6600640 1 -- 172.21.15.131:0/2827580118 shutdown_connections 2024-04-28T12:38:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.961+0000 7f85a6600640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f857c06c8f0 0x7f857c06edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.961+0000 7f85a6600640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85a0145ba0 0x7f85a013f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.961+0000 7f85a6600640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85a00d06e0 0x7f85a013ec00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.961+0000 7f85a6600640 1 --2- 172.21.15.131:0/2827580118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85a006f240 0x7f85a015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.961+0000 7f85a6600640 1 -- 172.21.15.131:0/2827580118 >> 172.21.15.131:0/2827580118 conn(0x7f85a00b2900 msgr2=0x7f85a0144b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:24.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.961+0000 7f85a6600640 1 -- 172.21.15.131:0/2827580118 shutdown_connections 2024-04-28T12:38:24.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:24.961+0000 7f85a6600640 1 -- 172.21.15.131:0/2827580118 wait complete. 2024-04-28T12:38:24.975 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:25.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.169+0000 7eff62a00640 1 Processor -- start 2024-04-28T12:38:25.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.169+0000 7eff62a00640 1 -- start start 2024-04-28T12:38:25.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.171+0000 7eff62a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7eff5c0cf440 0x7eff5c0b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:25.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.171+0000 7eff62a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5c06eaf0 con 0x7eff5c15d5f0 2024-04-28T12:38:25.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.171+0000 7eff62a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5c06edd0 con 0x7eff5c0cf820 2024-04-28T12:38:25.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.171+0000 7eff62a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7eff5c06f420 con 0x7eff5c0cf440 2024-04-28T12:38:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.171+0000 7eff5be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7eff5c0cf440 0x7eff5c0b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.172+0000 7eff60a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7eff5c15d5f0 0x7eff5c15da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43018/0 (socket says 172.21.15.131:43018) 2024-04-28T12:38:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.172+0000 7eff60a00640 1 -- 172.21.15.131:0/2678811711 learned_addr learned my addr 172.21.15.131:0/2678811711 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.172+0000 7eff5be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7eff5c0cf440 0x7eff5c0b5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54148/0 (socket says 172.21.15.131:54148) 2024-04-28T12:38:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.172+0000 7eff5be00640 1 -- 172.21.15.131:0/2678811711 >> v1:172.21.15.157:6789/0 conn(0x7eff5c0cf820 legacy=0x7eff5c0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.172+0000 7eff5be00640 1 -- 172.21.15.131:0/2678811711 >> v1:172.21.15.131:6789/0 conn(0x7eff5c15d5f0 legacy=0x7eff5c15da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.172+0000 7eff5be00640 1 -- 172.21.15.131:0/2678811711 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c06f0b0 con 0x7eff5c0cf440 2024-04-28T12:38:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.172+0000 7eff5b400640 1 -- 172.21.15.131:0/2678811711 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522902495 0 0) 0x7eff5c06eaf0 con 0x7eff5c15d5f0 2024-04-28T12:38:25.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.174+0000 7eff5be00640 1 --2- 172.21.15.131:0/2678811711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff5c0cf440 0x7eff5c0b5250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0097a0 tx=0x7eff4c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a989b7c0fad99be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:25.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.174+0000 7eff5b400640 1 -- 172.21.15.131:0/2678811711 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c037070 con 0x7eff5c0cf440 2024-04-28T12:38:25.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.174+0000 7eff5b400640 1 -- 172.21.15.131:0/2678811711 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff4c032440 con 0x7eff5c0cf440 2024-04-28T12:38:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.174+0000 7eff5b400640 1 -- 172.21.15.131:0/2678811711 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c032810 con 0x7eff5c0cf440 2024-04-28T12:38:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.174+0000 7eff62a00640 1 -- 172.21.15.131:0/2678811711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff5c0cf440 msgr2=0x7eff5c0b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.174+0000 7eff62a00640 1 --2- 172.21.15.131:0/2678811711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff5c0cf440 0x7eff5c0b5250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0097a0 tx=0x7eff4c02a1a0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.175+0000 7eff62a00640 1 -- 172.21.15.131:0/2678811711 shutdown_connections 2024-04-28T12:38:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.175+0000 7eff62a00640 1 --2- 172.21.15.131:0/2678811711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff5c0cf440 0x7eff5c0b5250 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.175+0000 7eff62a00640 1 -- 172.21.15.131:0/2678811711 >> 172.21.15.131:0/2678811711 conn(0x7eff5c0b2900 msgr2=0x7eff5c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.175+0000 7eff62a00640 1 -- 172.21.15.131:0/2678811711 shutdown_connections 2024-04-28T12:38:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.175+0000 7eff62a00640 1 -- 172.21.15.131:0/2678811711 wait complete. 2024-04-28T12:38:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.176+0000 7eff62a00640 1 Processor -- start 2024-04-28T12:38:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.177+0000 7eff62a00640 1 -- start start 2024-04-28T12:38:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.177+0000 7eff62a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff5c0cf440 0x7eff5c139b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.177+0000 7eff62a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff5c0cf820 0x7eff5c13d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.177+0000 7eff62a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff5c15d5f0 0x7eff5c13dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.177+0000 7eff62a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c0d0390 con 0x7eff5c0cf820 2024-04-28T12:38:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.177+0000 7eff62a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c06f8b0 con 0x7eff5c15d5f0 2024-04-28T12:38:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.178+0000 7eff62a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff5c0d0510 con 0x7eff5c0cf440 2024-04-28T12:38:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.178+0000 7eff53e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff5c0cf820 0x7eff5c13d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.178+0000 7eff53e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff5c0cf820 0x7eff5c13d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54162/0 (socket says 172.21.15.131:54162) 2024-04-28T12:38:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.178+0000 7eff53e00640 1 -- 172.21.15.131:0/3127490611 learned_addr learned my addr 172.21.15.131:0/3127490611 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.178+0000 7eff5be00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff5c0cf440 0x7eff5c139b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.178+0000 7eff60a00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff5c15d5f0 0x7eff5c13dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.178+0000 7eff53e00640 1 -- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff5c0cf440 msgr2=0x7eff5c139b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.178+0000 7eff53e00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff5c0cf440 0x7eff5c139b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.178+0000 7eff53e00640 1 -- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff5c15d5f0 msgr2=0x7eff5c13dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.178+0000 7eff53e00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff5c15d5f0 0x7eff5c13dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.178+0000 7eff53e00640 1 -- 172.21.15.131:0/3127490611 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c13e2f0 con 0x7eff5c0cf820 2024-04-28T12:38:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.179+0000 7eff53e00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff5c0cf820 0x7eff5c13d780 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7eff4400c430 tx=0x7eff4400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.179+0000 7eff58c00640 1 -- 172.21.15.131:0/3127490611 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44017070 con 0x7eff5c0cf820 2024-04-28T12:38:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.179+0000 7eff58c00640 1 -- 172.21.15.131:0/3127490611 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff440052d0 con 0x7eff5c0cf820 2024-04-28T12:38:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.179+0000 7eff58c00640 1 -- 172.21.15.131:0/3127490611 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44003830 con 0x7eff5c0cf820 2024-04-28T12:38:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.179+0000 7eff62a00640 1 -- 172.21.15.131:0/3127490611 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5c13e5d0 con 0x7eff5c0cf820 2024-04-28T12:38:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.179+0000 7eff62a00640 1 -- 172.21.15.131:0/3127490611 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5c13eb90 con 0x7eff5c0cf820 2024-04-28T12:38:25.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.180+0000 7eff62a00640 1 -- 172.21.15.131:0/3127490611 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff5c069350 con 0x7eff5c0cf820 2024-04-28T12:38:25.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.181+0000 7eff58c00640 1 -- 172.21.15.131:0/3127490611 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff4401a070 con 0x7eff5c0cf820 2024-04-28T12:38:25.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.182+0000 7eff58c00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7eff3806c8f0 0x7eff3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:25.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.182+0000 7eff58c00640 1 -- 172.21.15.131:0/3127490611 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7eff4408b2c0 con 0x7eff5c0cf820 2024-04-28T12:38:25.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.183+0000 7eff5be00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7eff3806c8f0 0x7eff3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:25.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.184+0000 7eff5be00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7eff3806c8f0 0x7eff3806edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7eff4c034000 tx=0x7eff4c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:25.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.186+0000 7eff58c00640 1 -- 172.21.15.131:0/3127490611 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff440592e0 con 0x7eff5c0cf820 2024-04-28T12:38:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.374+0000 7eff62a00640 1 -- 172.21.15.131:0/3127490611 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7eff5c0b1c50 con 0x7eff5c0cf820 2024-04-28T12:38:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.375+0000 7eff58c00640 1 -- 172.21.15.131:0/3127490611 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7eff44058be0 con 0x7eff5c0cf820 2024-04-28T12:38:25.372 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.377+0000 7eff62a00640 1 -- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7eff3806c8f0 msgr2=0x7eff3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.377+0000 7eff62a00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7eff3806c8f0 0x7eff3806edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7eff4c034000 tx=0x7eff4c02e000 comp rx=0 tx=0).stop 2024-04-28T12:38:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.377+0000 7eff62a00640 1 -- 172.21.15.131:0/3127490611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff5c0cf820 msgr2=0x7eff5c13d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.377+0000 7eff62a00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff5c0cf820 0x7eff5c13d780 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7eff4400c430 tx=0x7eff4400c8f0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.377+0000 7eff62a00640 1 -- 172.21.15.131:0/3127490611 shutdown_connections 2024-04-28T12:38:25.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.377+0000 7eff62a00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7eff3806c8f0 0x7eff3806edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.377+0000 7eff62a00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff5c15d5f0 0x7eff5c13dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.377+0000 7eff62a00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff5c0cf820 0x7eff5c13d780 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.377+0000 7eff62a00640 1 --2- 172.21.15.131:0/3127490611 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff5c0cf440 0x7eff5c139b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.378+0000 7eff62a00640 1 -- 172.21.15.131:0/3127490611 >> 172.21.15.131:0/3127490611 conn(0x7eff5c0b2900 msgr2=0x7eff5c144180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.378+0000 7eff62a00640 1 -- 172.21.15.131:0/3127490611 shutdown_connections 2024-04-28T12:38:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.378+0000 7eff62a00640 1 -- 172.21.15.131:0/3127490611 wait complete. 2024-04-28T12:38:25.391 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:25.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.585+0000 7f1dfae00640 1 Processor -- start 2024-04-28T12:38:25.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.585+0000 7f1dfae00640 1 -- start start 2024-04-28T12:38:25.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.587+0000 7f1dfae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df41572b0 0x7f1df40b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:25.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.587+0000 7f1dfae00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1df40affb0 0x7f1df40d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:25.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.587+0000 7f1dfae00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1df415d2e0 con 0x7f1df40d0bd0 2024-04-28T12:38:25.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.587+0000 7f1dfae00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1df415d5c0 con 0x7f1df40affb0 2024-04-28T12:38:25.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.587+0000 7f1dfae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1df415d8a0 con 0x7f1df41572b0 2024-04-28T12:38:25.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.587+0000 7f1dfa400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1df40affb0 0x7f1df40d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:25.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.587+0000 7f1dfa400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1df40affb0 0x7f1df40d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54170/0 (socket says 172.21.15.131:54170) 2024-04-28T12:38:25.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.587+0000 7f1dfa400640 1 -- 172.21.15.131:0/820536193 learned_addr learned my addr 172.21.15.131:0/820536193 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:25.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.587+0000 7f1df9a00640 1 --2- 172.21.15.131:0/820536193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df41572b0 0x7f1df40b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:25.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.588+0000 7f1dfa400640 1 -- 172.21.15.131:0/820536193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df41572b0 msgr2=0x7f1df40b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:25.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.588+0000 7f1dfa400640 1 --2- 172.21.15.131:0/820536193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df41572b0 0x7f1df40b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.588+0000 7f1dfa400640 1 -- 172.21.15.131:0/820536193 >> v1:172.21.15.157:6789/0 conn(0x7f1df40d0bd0 legacy=0x7f1df40af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.588+0000 7f1dfa400640 1 -- 172.21.15.131:0/820536193 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df415db80 con 0x7f1df40affb0 2024-04-28T12:38:25.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.589+0000 7f1dfa400640 1 --2- 172.21.15.131:0/820536193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1df40affb0 0x7f1df40d1050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009e50 tx=0x7f1dec02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8236a01297b60df5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:25.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.589+0000 7f1df3e00640 1 -- 172.21.15.131:0/820536193 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec036070 con 0x7f1df40affb0 2024-04-28T12:38:25.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.589+0000 7f1df3e00640 1 -- 172.21.15.131:0/820536193 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dec031600 con 0x7f1df40affb0 2024-04-28T12:38:25.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.589+0000 7f1df3e00640 1 -- 172.21.15.131:0/820536193 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0319d0 con 0x7f1df40affb0 2024-04-28T12:38:25.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.590+0000 7f1dfae00640 1 -- 172.21.15.131:0/820536193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1df40affb0 msgr2=0x7f1df40d1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.590+0000 7f1dfae00640 1 --2- 172.21.15.131:0/820536193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1df40affb0 0x7f1df40d1050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009e50 tx=0x7f1dec02a5b0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.590+0000 7f1dfae00640 1 -- 172.21.15.131:0/820536193 shutdown_connections 2024-04-28T12:38:25.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.590+0000 7f1dfae00640 1 --2- 172.21.15.131:0/820536193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1df40affb0 0x7f1df40d1050 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.590+0000 7f1dfae00640 1 --2- 172.21.15.131:0/820536193 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df41572b0 0x7f1df40b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.590+0000 7f1dfae00640 1 -- 172.21.15.131:0/820536193 >> 172.21.15.131:0/820536193 conn(0x7f1df40684e0 msgr2=0x7f1df40688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:25.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.590+0000 7f1dfae00640 1 -- 172.21.15.131:0/820536193 shutdown_connections 2024-04-28T12:38:25.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.590+0000 7f1dfae00640 1 -- 172.21.15.131:0/820536193 wait complete. 2024-04-28T12:38:25.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.591+0000 7f1dfae00640 1 Processor -- start 2024-04-28T12:38:25.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.592+0000 7f1dfae00640 1 -- start start 2024-04-28T12:38:25.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.592+0000 7f1dfae00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1df40affb0 0x7f1df40af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:25.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.592+0000 7f1dfae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df40d0bd0 0x7f1df414c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:25.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.592+0000 7f1dfae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df41572b0 0x7f1df415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:25.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.592+0000 7f1dfae00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1df4160c00 con 0x7f1df40affb0 2024-04-28T12:38:25.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.592+0000 7f1dfae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1df4160a80 con 0x7f1df41572b0 2024-04-28T12:38:25.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.592+0000 7f1dfae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1df4160d80 con 0x7f1df40d0bd0 2024-04-28T12:38:25.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.593+0000 7f1df9a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1df40affb0 0x7f1df40af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:25.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.593+0000 7f1df9a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1df40affb0 0x7f1df40af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54184/0 (socket says 172.21.15.131:54184) 2024-04-28T12:38:25.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.593+0000 7f1df9a00640 1 -- 172.21.15.131:0/2041868330 learned_addr learned my addr 172.21.15.131:0/2041868330 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.593+0000 7f1dfa400640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df41572b0 0x7f1df415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.593+0000 7f1df9000640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df40d0bd0 0x7f1df414c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.593+0000 7f1df9a00640 1 -- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df40d0bd0 msgr2=0x7f1df414c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.593+0000 7f1df9a00640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df40d0bd0 0x7f1df414c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.593+0000 7f1df9a00640 1 -- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df41572b0 msgr2=0x7f1df415b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.593+0000 7f1df9a00640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df41572b0 0x7f1df415b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.593+0000 7f1df9a00640 1 -- 172.21.15.131:0/2041868330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df414c610 con 0x7f1df40affb0 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.593+0000 7f1df9000640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df40d0bd0 0x7f1df414c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.593+0000 7f1dfa400640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df41572b0 0x7f1df415b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.594+0000 7f1df9a00640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1df40affb0 0x7f1df40af080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1de800d5f0 tx=0x7f1de800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.594+0000 7f1df1600640 1 -- 172.21.15.131:0/2041868330 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de80125f0 con 0x7f1df40affb0 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.594+0000 7f1df1600640 1 -- 172.21.15.131:0/2041868330 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de8019070 con 0x7f1df40affb0 2024-04-28T12:38:25.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.594+0000 7f1df1600640 1 -- 172.21.15.131:0/2041868330 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de80129d0 con 0x7f1df40affb0 2024-04-28T12:38:25.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.594+0000 7f1dfae00640 1 -- 172.21.15.131:0/2041868330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df415b920 con 0x7f1df40affb0 2024-04-28T12:38:25.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.594+0000 7f1dfae00640 1 -- 172.21.15.131:0/2041868330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df415bee0 con 0x7f1df40affb0 2024-04-28T12:38:25.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.595+0000 7f1dfae00640 1 -- 172.21.15.131:0/2041868330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df40506c0 con 0x7f1df40affb0 2024-04-28T12:38:25.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.596+0000 7f1df1600640 1 -- 172.21.15.131:0/2041868330 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1de8012b70 con 0x7f1df40affb0 2024-04-28T12:38:25.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.597+0000 7f1df1600640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1dd406c8f0 0x7f1dd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:25.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.597+0000 7f1df1600640 1 -- 172.21.15.131:0/2041868330 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1de808c3b0 con 0x7f1df40affb0 2024-04-28T12:38:25.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.598+0000 7f1df9000640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1dd406c8f0 0x7f1dd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:25.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.599+0000 7f1df9000640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1dd406c8f0 0x7f1dd406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1df415ca30 tx=0x7f1de0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:25.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.602+0000 7f1df1600640 1 -- 172.21.15.131:0/2041868330 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1de805a340 con 0x7f1df40affb0 2024-04-28T12:38:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.786+0000 7f1dfae00640 1 -- 172.21.15.131:0/2041868330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1df415e520 con 0x7f1df40affb0 2024-04-28T12:38:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.786+0000 7f1df1600640 1 -- 172.21.15.131:0/2041868330 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f1de8059c40 con 0x7f1df40affb0 2024-04-28T12:38:25.783 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.789+0000 7f1dfae00640 1 -- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1dd406c8f0 msgr2=0x7f1dd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.789+0000 7f1dfae00640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1dd406c8f0 0x7f1dd406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1df415ca30 tx=0x7f1de0009040 comp rx=0 tx=0).stop 2024-04-28T12:38:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.789+0000 7f1dfae00640 1 -- 172.21.15.131:0/2041868330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1df40affb0 msgr2=0x7f1df40af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.789+0000 7f1dfae00640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1df40affb0 0x7f1df40af080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1de800d5f0 tx=0x7f1de800dab0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.789+0000 7f1dfae00640 1 -- 172.21.15.131:0/2041868330 shutdown_connections 2024-04-28T12:38:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.789+0000 7f1dfae00640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1dd406c8f0 0x7f1dd406edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.789+0000 7f1dfae00640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df41572b0 0x7f1df415b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.789+0000 7f1dfae00640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1df40d0bd0 0x7f1df414c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.789+0000 7f1dfae00640 1 --2- 172.21.15.131:0/2041868330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1df40affb0 0x7f1df40af080 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:25.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.789+0000 7f1dfae00640 1 -- 172.21.15.131:0/2041868330 >> 172.21.15.131:0/2041868330 conn(0x7f1df40684e0 msgr2=0x7f1df406f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:25.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.789+0000 7f1dfae00640 1 -- 172.21.15.131:0/2041868330 shutdown_connections 2024-04-28T12:38:25.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:25.789+0000 7f1dfae00640 1 -- 172.21.15.131:0/2041868330 wait complete. 2024-04-28T12:38:25.803 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:25.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.002+0000 7f09bf200640 1 Processor -- start 2024-04-28T12:38:25.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.002+0000 7f09bf200640 1 -- start start 2024-04-28T12:38:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.004+0000 7f09bf200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f09c00d3830 0x7f09c00d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.004+0000 7f09bf200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09c00b3cb0 0x7f09c00b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.004+0000 7f09bf200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09c00d2d70 con 0x7f09c015f180 2024-04-28T12:38:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.004+0000 7f09bf200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f09c015f560 con 0x7f09c00b3cb0 2024-04-28T12:38:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.004+0000 7f09bf200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f09c00d06e0 con 0x7f09c00d3830 2024-04-28T12:38:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.004+0000 7f09be800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09c00b3cb0 0x7f09c00b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.004+0000 7f09be800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f09c00b3cb0 0x7f09c00b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54190/0 (socket says 172.21.15.131:54190) 2024-04-28T12:38:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.004+0000 7f09be800640 1 -- 172.21.15.131:0/3549099618 learned_addr learned my addr 172.21.15.131:0/3549099618 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.004+0000 7f09be800640 1 -- 172.21.15.131:0/3549099618 >> v2:172.21.15.157:3300/0 conn(0x7f09c00d3830 msgr2=0x7f09c00d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.004+0000 7f09be800640 1 --2- 172.21.15.131:0/3549099618 >> v2:172.21.15.157:3300/0 conn(0x7f09c00d3830 0x7f09c00d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.004+0000 7f09be800640 1 -- 172.21.15.131:0/3549099618 >> v1:172.21.15.157:6789/0 conn(0x7f09c015f180 legacy=0x7f09c00d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:26.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.004+0000 7f09be800640 1 -- 172.21.15.131:0/3549099618 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c00d09c0 con 0x7f09c00b3cb0 2024-04-28T12:38:26.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.006+0000 7f09be800640 1 --2- 172.21.15.131:0/3549099618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c00b3cb0 0x7f09c00b1740 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f09c00b77c0 tx=0x7f09b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb2007f41b676fa server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:26.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.006+0000 7f09bca00640 1 -- 172.21.15.131:0/3549099618 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8037070 con 0x7f09c00b3cb0 2024-04-28T12:38:26.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.006+0000 7f09bca00640 1 -- 172.21.15.131:0/3549099618 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b8032440 con 0x7f09c00b3cb0 2024-04-28T12:38:26.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.006+0000 7f09bca00640 1 -- 172.21.15.131:0/3549099618 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8032810 con 0x7f09c00b3cb0 2024-04-28T12:38:26.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.006+0000 7f09bf200640 1 -- 172.21.15.131:0/3549099618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c00b3cb0 msgr2=0x7f09c00b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.006+0000 7f09bf200640 1 --2- 172.21.15.131:0/3549099618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c00b3cb0 0x7f09c00b1740 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f09c00b77c0 tx=0x7f09b802a230 comp rx=0 tx=0).stop 2024-04-28T12:38:26.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.007+0000 7f09bf200640 1 -- 172.21.15.131:0/3549099618 shutdown_connections 2024-04-28T12:38:26.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.007+0000 7f09bf200640 1 --2- 172.21.15.131:0/3549099618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c00b3cb0 0x7f09c00b1740 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.007+0000 7f09bf200640 1 --2- 172.21.15.131:0/3549099618 >> v2:172.21.15.157:3300/0 conn(0x7f09c00d3830 0x7f09c00d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.007+0000 7f09bf200640 1 -- 172.21.15.131:0/3549099618 >> 172.21.15.131:0/3549099618 conn(0x7f09c00d4bd0 msgr2=0x7f09c00d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:26.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.007+0000 7f09bf200640 1 -- 172.21.15.131:0/3549099618 shutdown_connections 2024-04-28T12:38:26.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.007+0000 7f09bf200640 1 -- 172.21.15.131:0/3549099618 wait complete. 2024-04-28T12:38:26.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.008+0000 7f09bf200640 1 Processor -- start 2024-04-28T12:38:26.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.008+0000 7f09bf200640 1 -- start start 2024-04-28T12:38:26.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.009+0000 7f09bf200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09c00b3cb0 0x7f09c01531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.009+0000 7f09bf200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c00d3830 0x7f09c00b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.009+0000 7f09bf200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c015f180 0x7f09c0154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.009+0000 7f09bf200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f09c00d11c0 con 0x7f09c015f180 2024-04-28T12:38:26.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.009+0000 7f09bf200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f09c00d2a40 con 0x7f09c00d3830 2024-04-28T12:38:26.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.009+0000 7f09bf200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f09c0153720 con 0x7f09c00b3cb0 2024-04-28T12:38:26.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.010+0000 7f09be800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c015f180 0x7f09c0154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.010+0000 7f09be800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c015f180 0x7f09c0154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54202/0 (socket says 172.21.15.131:54202) 2024-04-28T12:38:26.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.010+0000 7f09be800640 1 -- 172.21.15.131:0/2205636477 learned_addr learned my addr 172.21.15.131:0/2205636477 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:26.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.010+0000 7f09be800640 1 -- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09c00b3cb0 msgr2=0x7f09c01531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.010+0000 7f09be800640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09c00b3cb0 0x7f09c01531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.010+0000 7f09be800640 1 -- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c00d3830 msgr2=0x7f09c00b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.010+0000 7f09bd400640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c00d3830 0x7f09c00b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.010+0000 7f09be800640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c00d3830 0x7f09c00b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.010+0000 7f09be800640 1 -- 172.21.15.131:0/2205636477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c0158060 con 0x7f09c015f180 2024-04-28T12:38:26.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.010+0000 7f09bd400640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c00d3830 0x7f09c00b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:26.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.010+0000 7f09be800640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c015f180 0x7f09c0154ae0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f09b80090b0 tx=0x7f09b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:26.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.011+0000 7f09b6000640 1 -- 172.21.15.131:0/2205636477 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8041070 con 0x7f09c015f180 2024-04-28T12:38:26.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.011+0000 7f09b6000640 1 -- 172.21.15.131:0/2205636477 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b803c950 con 0x7f09c015f180 2024-04-28T12:38:26.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.011+0000 7f09b6000640 1 -- 172.21.15.131:0/2205636477 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8037040 con 0x7f09c015f180 2024-04-28T12:38:26.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.011+0000 7f09bf200640 1 -- 172.21.15.131:0/2205636477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c0158340 con 0x7f09c015f180 2024-04-28T12:38:26.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.011+0000 7f09bf200640 1 -- 172.21.15.131:0/2205636477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c0158900 con 0x7f09c015f180 2024-04-28T12:38:26.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.012+0000 7f09bf200640 1 -- 172.21.15.131:0/2205636477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c00506c0 con 0x7f09c015f180 2024-04-28T12:38:26.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.013+0000 7f09b6000640 1 -- 172.21.15.131:0/2205636477 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09b8044070 con 0x7f09c015f180 2024-04-28T12:38:26.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.014+0000 7f09b6000640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f099c06c8f0 0x7f099c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.014+0000 7f09b6000640 1 -- 172.21.15.131:0/2205636477 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f09b80aca00 con 0x7f09c015f180 2024-04-28T12:38:26.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.014+0000 7f09bde00640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f099c06c8f0 0x7f099c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.015+0000 7f09bde00640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f099c06c8f0 0x7f099c06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f09c0155fb0 tx=0x7f09b0006ba0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:26.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.018+0000 7f09b6000640 1 -- 172.21.15.131:0/2205636477 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f09b807aa20 con 0x7f09c015f180 2024-04-28T12:38:26.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.218+0000 7f09bf200640 1 -- 172.21.15.131:0/2205636477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f09c00b2a20 con 0x7f09c015f180 2024-04-28T12:38:26.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.219+0000 7f09b6000640 1 -- 172.21.15.131:0/2205636477 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f09b807a320 con 0x7f09c015f180 2024-04-28T12:38:26.216 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:26.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.221+0000 7f09bf200640 1 -- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f099c06c8f0 msgr2=0x7f099c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.221+0000 7f09bf200640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f099c06c8f0 0x7f099c06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f09c0155fb0 tx=0x7f09b0006ba0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.221+0000 7f09bf200640 1 -- 172.21.15.131:0/2205636477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c015f180 msgr2=0x7f09c0154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.221+0000 7f09bf200640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c015f180 0x7f09c0154ae0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f09b80090b0 tx=0x7f09b80046c0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.222+0000 7f09bf200640 1 -- 172.21.15.131:0/2205636477 shutdown_connections 2024-04-28T12:38:26.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.222+0000 7f09bf200640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f099c06c8f0 0x7f099c06edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.222+0000 7f09bf200640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09c015f180 0x7f09c0154ae0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.222+0000 7f09bf200640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09c00d3830 0x7f09c00b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.222+0000 7f09bf200640 1 --2- 172.21.15.131:0/2205636477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09c00b3cb0 0x7f09c01531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.222+0000 7f09bf200640 1 -- 172.21.15.131:0/2205636477 >> 172.21.15.131:0/2205636477 conn(0x7f09c00d4bd0 msgr2=0x7f09c00d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:26.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.222+0000 7f09bf200640 1 -- 172.21.15.131:0/2205636477 shutdown_connections 2024-04-28T12:38:26.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.222+0000 7f09bf200640 1 -- 172.21.15.131:0/2205636477 wait complete. 2024-04-28T12:38:26.247 INFO:tasks.cephfs_test_runner:Starting test: test_acls (tasks.cephfs.test_acls.TestACLs) 2024-04-28T12:38:26.248 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:26.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.452+0000 7f9360c00640 1 Processor -- start 2024-04-28T12:38:26.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.452+0000 7f9360c00640 1 -- start start 2024-04-28T12:38:26.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.454+0000 7f9360c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f935c1628e0 0x7f935c162d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.454+0000 7f9360c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935c0af720 0x7f935c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.454+0000 7f9360c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935c145710 con 0x7f935c145330 2024-04-28T12:38:26.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.454+0000 7f9360c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f935c073dc0 con 0x7f935c1628e0 2024-04-28T12:38:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.454+0000 7f9360c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f935c0ae960 con 0x7f935c0af720 2024-04-28T12:38:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.455+0000 7f935be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935c0af720 0x7f935c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.455+0000 7f935be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935c0af720 0x7f935c0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:36968/0 (socket says 172.21.15.131:36968) 2024-04-28T12:38:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.455+0000 7f935be00640 1 -- 172.21.15.131:0/4294166040 learned_addr learned my addr 172.21.15.131:0/4294166040 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:26.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.455+0000 7f935aa00640 1 --2- 172.21.15.131:0/4294166040 >> v2:172.21.15.157:3300/0 conn(0x7f935c1628e0 0x7f935c162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.455+0000 7f935be00640 1 -- 172.21.15.131:0/4294166040 >> v2:172.21.15.157:3300/0 conn(0x7f935c1628e0 msgr2=0x7f935c162d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:26.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.455+0000 7f935be00640 1 --2- 172.21.15.131:0/4294166040 >> v2:172.21.15.157:3300/0 conn(0x7f935c1628e0 0x7f935c162d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.455+0000 7f935be00640 1 -- 172.21.15.131:0/4294166040 >> v1:172.21.15.157:6789/0 conn(0x7f935c145330 legacy=0x7f935c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.455+0000 7f935be00640 1 -- 172.21.15.131:0/4294166040 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c0aec40 con 0x7f935c0af720 2024-04-28T12:38:26.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.457+0000 7f935be00640 1 --2- 172.21.15.131:0/4294166040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935c0af720 0x7f935c0afb90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9354009e50 tx=0x7f935402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c62b15015458cab server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:26.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.458+0000 7f935a000640 1 -- 172.21.15.131:0/4294166040 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354036070 con 0x7f935c0af720 2024-04-28T12:38:26.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.458+0000 7f935a000640 1 -- 172.21.15.131:0/4294166040 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9354031600 con 0x7f935c0af720 2024-04-28T12:38:26.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.458+0000 7f935a000640 1 -- 172.21.15.131:0/4294166040 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93540319d0 con 0x7f935c0af720 2024-04-28T12:38:26.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.458+0000 7f9360c00640 1 -- 172.21.15.131:0/4294166040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935c0af720 msgr2=0x7f935c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.458+0000 7f9360c00640 1 --2- 172.21.15.131:0/4294166040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935c0af720 0x7f935c0afb90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9354009e50 tx=0x7f935402a710 comp rx=0 tx=0).stop 2024-04-28T12:38:26.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.459+0000 7f9360c00640 1 -- 172.21.15.131:0/4294166040 shutdown_connections 2024-04-28T12:38:26.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.459+0000 7f9360c00640 1 --2- 172.21.15.131:0/4294166040 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935c0af720 0x7f935c0afb90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.459+0000 7f9360c00640 1 --2- 172.21.15.131:0/4294166040 >> v2:172.21.15.157:3300/0 conn(0x7f935c1628e0 0x7f935c162d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.459+0000 7f9360c00640 1 -- 172.21.15.131:0/4294166040 >> 172.21.15.131:0/4294166040 conn(0x7f935c068090 msgr2=0x7f935c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:26.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.459+0000 7f9360c00640 1 -- 172.21.15.131:0/4294166040 shutdown_connections 2024-04-28T12:38:26.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.459+0000 7f9360c00640 1 -- 172.21.15.131:0/4294166040 wait complete. 2024-04-28T12:38:26.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.461+0000 7f9360c00640 1 Processor -- start 2024-04-28T12:38:26.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.461+0000 7f9360c00640 1 -- start start 2024-04-28T12:38:26.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.462+0000 7f9360c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935c0af720 0x7f935c0cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.462+0000 7f9360c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f935c145330 0x7f935c0b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.462+0000 7f9360c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935c1628e0 0x7f935c1410d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.462+0000 7f9360c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f935c0b00d0 con 0x7f935c145330 2024-04-28T12:38:26.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.462+0000 7f9360c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f935c0af200 con 0x7f935c1628e0 2024-04-28T12:38:26.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.462+0000 7f9360c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f935c0b0250 con 0x7f935c0af720 2024-04-28T12:38:26.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.463+0000 7f935aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f935c145330 0x7f935c0b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.463+0000 7f935aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f935c145330 0x7f935c0b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54208/0 (socket says 172.21.15.131:54208) 2024-04-28T12:38:26.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.463+0000 7f935aa00640 1 -- 172.21.15.131:0/2508963909 learned_addr learned my addr 172.21.15.131:0/2508963909 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:26.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.463+0000 7f935b400640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935c0af720 0x7f935c0cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.463+0000 7f935be00640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935c1628e0 0x7f935c1410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.463+0000 7f935aa00640 1 -- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935c0af720 msgr2=0x7f935c0cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.463+0000 7f935aa00640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935c0af720 0x7f935c0cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.463+0000 7f935aa00640 1 -- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935c1628e0 msgr2=0x7f935c1410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.463+0000 7f935aa00640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935c1628e0 0x7f935c1410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.463+0000 7f935aa00640 1 -- 172.21.15.131:0/2508963909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c0b1860 con 0x7f935c145330 2024-04-28T12:38:26.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.464+0000 7f935aa00640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f935c145330 0x7f935c0b2d70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f934c00e8a0 tx=0x7f934c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.464+0000 7f9353400640 1 -- 172.21.15.131:0/2508963909 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c017070 con 0x7f935c145330 2024-04-28T12:38:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.464+0000 7f9353400640 1 -- 172.21.15.131:0/2508963909 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f934c00c8e0 con 0x7f935c145330 2024-04-28T12:38:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.464+0000 7f9360c00640 1 -- 172.21.15.131:0/2508963909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935c0b1b40 con 0x7f935c145330 2024-04-28T12:38:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.464+0000 7f9353400640 1 -- 172.21.15.131:0/2508963909 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c00ccd0 con 0x7f935c145330 2024-04-28T12:38:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.464+0000 7f9360c00640 1 -- 172.21.15.131:0/2508963909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935c0b2570 con 0x7f935c145330 2024-04-28T12:38:26.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.465+0000 7f9360c00640 1 -- 172.21.15.131:0/2508963909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935c069350 con 0x7f935c145330 2024-04-28T12:38:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.466+0000 7f9353400640 1 -- 172.21.15.131:0/2508963909 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f934c01a070 con 0x7f935c145330 2024-04-28T12:38:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.467+0000 7f9353400640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f933806c8f0 0x7f933806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.467+0000 7f9353400640 1 -- 172.21.15.131:0/2508963909 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f934c08c230 con 0x7f935c145330 2024-04-28T12:38:26.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.468+0000 7f935b400640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f933806c8f0 0x7f933806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.469+0000 7f935b400640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f933806c8f0 0x7f933806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9344000f60 tx=0x7f9344009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:26.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.472+0000 7f9353400640 1 -- 172.21.15.131:0/2508963909 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f934c05a140 con 0x7f935c145330 2024-04-28T12:38:26.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.673+0000 7f9360c00640 1 -- 172.21.15.131:0/2508963909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f935c15f9f0 con 0x7f935c145330 2024-04-28T12:38:26.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.676+0000 7f9353400640 1 -- 172.21.15.131:0/2508963909 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7f934c0790e0 con 0x7f935c145330 2024-04-28T12:38:26.674 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:26.674 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:26.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.679+0000 7f9360c00640 1 -- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f933806c8f0 msgr2=0x7f933806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.679+0000 7f9360c00640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f933806c8f0 0x7f933806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9344000f60 tx=0x7f9344009040 comp rx=0 tx=0).stop 2024-04-28T12:38:26.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.679+0000 7f9360c00640 1 -- 172.21.15.131:0/2508963909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f935c145330 msgr2=0x7f935c0b2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.679+0000 7f9360c00640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f935c145330 0x7f935c0b2d70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f934c00e8a0 tx=0x7f934c00ed60 comp rx=0 tx=0).stop 2024-04-28T12:38:26.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.679+0000 7f9360c00640 1 -- 172.21.15.131:0/2508963909 shutdown_connections 2024-04-28T12:38:26.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.679+0000 7f9360c00640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f933806c8f0 0x7f933806edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.679+0000 7f9360c00640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f935c1628e0 0x7f935c1410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.679+0000 7f9360c00640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f935c145330 0x7f935c0b2d70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.679+0000 7f9360c00640 1 --2- 172.21.15.131:0/2508963909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f935c0af720 0x7f935c0cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.679+0000 7f9360c00640 1 -- 172.21.15.131:0/2508963909 >> 172.21.15.131:0/2508963909 conn(0x7f935c068090 msgr2=0x7f935c15eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:26.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.679+0000 7f9360c00640 1 -- 172.21.15.131:0/2508963909 shutdown_connections 2024-04-28T12:38:26.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.679+0000 7f9360c00640 1 -- 172.21.15.131:0/2508963909 wait complete. 2024-04-28T12:38:26.696 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:26.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:26.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.902+0000 7f3dab200640 1 Processor -- start 2024-04-28T12:38:26.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.903+0000 7f3dab200640 1 -- start start 2024-04-28T12:38:26.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.904+0000 7f3dab200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dac15a7e0 0x7f3dac0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.904+0000 7f3dab200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dac0b3900 con 0x7f3dac0b5410 2024-04-28T12:38:26.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.904+0000 7f3dab200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dac0b3b70 con 0x7f3dac15d9c0 2024-04-28T12:38:26.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.904+0000 7f3dab200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3dac15abc0 con 0x7f3dac15a7e0 2024-04-28T12:38:26.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.904+0000 7f3da1400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3dac0b5410 0x7f3dac15d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43032/0 (socket says 172.21.15.131:43032) 2024-04-28T12:38:26.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.904+0000 7f3da1400640 1 -- 172.21.15.131:0/758096264 learned_addr learned my addr 172.21.15.131:0/758096264 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:26.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9e00640 1 --2- 172.21.15.131:0/758096264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dac15a7e0 0x7f3dac0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3437871145 0 0) 0x7f3dac0b3900 con 0x7f3dac0b5410 2024-04-28T12:38:26.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d90003520 con 0x7f3dac0b5410 2024-04-28T12:38:26.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3893434885 0 0) 0x7f3d90003520 con 0x7f3dac0b5410 2024-04-28T12:38:26.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dac15a7e0 msgr2=0x7f3dac0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:26.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 --2- 172.21.15.131:0/758096264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dac15a7e0 0x7f3dac0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 >> v1:172.21.15.157:6789/0 conn(0x7f3dac15d9c0 legacy=0x7f3dac15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac0b3e50 con 0x7f3dac0b5410 2024-04-28T12:38:26.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f3d94001670 con 0x7f3dac0b5410 2024-04-28T12:38:26.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 >> v1:172.21.15.131:6789/0 conn(0x7f3dac0b5410 legacy=0x7f3dac15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 --2- 172.21.15.131:0/758096264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dac15a7e0 0x7f3d90003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 --2- 172.21.15.131:0/758096264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d90004080 0x7f3d90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 --2- 172.21.15.131:0/758096264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d90004d90 0x7f3d90005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac15abc0 con 0x7f3d90004d90 2024-04-28T12:38:26.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9000bbd0 con 0x7f3d90004080 2024-04-28T12:38:26.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3d9000bdd0 con 0x7f3dac15a7e0 2024-04-28T12:38:26.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.905+0000 7f3daa800640 1 --2- 172.21.15.131:0/758096264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d90004d90 0x7f3d90005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.906+0000 7f3da1400640 1 --2- 172.21.15.131:0/758096264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d90004080 0x7f3d90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.906+0000 7f3da9e00640 1 --2- 172.21.15.131:0/758096264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dac15a7e0 0x7f3d90003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.906+0000 7f3daa800640 1 -- 172.21.15.131:0/758096264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dac15a7e0 msgr2=0x7f3d90003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:26.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.906+0000 7f3daa800640 1 --2- 172.21.15.131:0/758096264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dac15a7e0 0x7f3d90003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.906+0000 7f3daa800640 1 -- 172.21.15.131:0/758096264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d90004080 msgr2=0x7f3d90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:26.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.906+0000 7f3daa800640 1 --2- 172.21.15.131:0/758096264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d90004080 0x7f3d90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.906+0000 7f3daa800640 1 -- 172.21.15.131:0/758096264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d90003520 con 0x7f3d90004d90 2024-04-28T12:38:26.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.907+0000 7f3daa800640 1 --2- 172.21.15.131:0/758096264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d90004d90 0x7f3d90005210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3da4001e50 tx=0x7f3d90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6fa2f5c4ac6da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:26.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.907+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4001a70 con 0x7f3d90004d90 2024-04-28T12:38:26.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.907+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da4003970 con 0x7f3d90004d90 2024-04-28T12:38:26.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.907+0000 7f3da9400640 1 -- 172.21.15.131:0/758096264 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4003d60 con 0x7f3d90004d90 2024-04-28T12:38:26.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.908+0000 7f3dab200640 1 -- 172.21.15.131:0/758096264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d90004d90 msgr2=0x7f3d90005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.908+0000 7f3dab200640 1 --2- 172.21.15.131:0/758096264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d90004d90 0x7f3d90005210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3da4001e50 tx=0x7f3d90005830 comp rx=0 tx=0).stop 2024-04-28T12:38:26.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.908+0000 7f3daa800640 1 -- 172.21.15.131:0/758096264 reap_dead start 2024-04-28T12:38:26.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.908+0000 7f3dab200640 1 -- 172.21.15.131:0/758096264 shutdown_connections 2024-04-28T12:38:26.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.908+0000 7f3dab200640 1 -- 172.21.15.131:0/758096264 >> 172.21.15.131:0/758096264 conn(0x7f3dac068090 msgr2=0x7f3dac0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:26.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.908+0000 7f3dab200640 1 -- 172.21.15.131:0/758096264 shutdown_connections 2024-04-28T12:38:26.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.908+0000 7f3dab200640 1 -- 172.21.15.131:0/758096264 wait complete. 2024-04-28T12:38:26.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.910+0000 7f3dab200640 1 Processor -- start 2024-04-28T12:38:26.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.910+0000 7f3dab200640 1 -- start start 2024-04-28T12:38:26.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3dab200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d90004d90 0x7f3dac1e6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3dab200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac1e50f0 0x7f3dac157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3dab200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac154230 0x7f3dac157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3dab200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac15e9f0 con 0x7f3dac154230 2024-04-28T12:38:26.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3dab200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac15e010 con 0x7f3dac1e50f0 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3dab200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3dac15eb70 con 0x7f3d90004d90 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3daa800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac154230 0x7f3dac157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3daa800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac154230 0x7f3dac157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54228/0 (socket says 172.21.15.131:54228) 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3daa800640 1 -- 172.21.15.131:0/2274111420 learned_addr learned my addr 172.21.15.131:0/2274111420 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3da1400640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac1e50f0 0x7f3dac157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3da9e00640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d90004d90 0x7f3dac1e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3daa800640 1 -- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d90004d90 msgr2=0x7f3dac1e6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3daa800640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d90004d90 0x7f3dac1e6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3daa800640 1 -- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac1e50f0 msgr2=0x7f3dac157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3daa800640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac1e50f0 0x7f3dac157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.911+0000 7f3daa800640 1 -- 172.21.15.131:0/2274111420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac1e5570 con 0x7f3dac154230 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.912+0000 7f3daa800640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac154230 0x7f3dac157940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3da4008fd0 tx=0x7f3da40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.912+0000 7f3da2a00640 1 -- 172.21.15.131:0/2274111420 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4037040 con 0x7f3dac154230 2024-04-28T12:38:26.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.912+0000 7f3da2a00640 1 -- 172.21.15.131:0/2274111420 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da4003e50 con 0x7f3dac154230 2024-04-28T12:38:26.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.912+0000 7f3da2a00640 1 -- 172.21.15.131:0/2274111420 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4007db0 con 0x7f3dac154230 2024-04-28T12:38:26.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.912+0000 7f3dab200640 1 -- 172.21.15.131:0/2274111420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dac1e57f0 con 0x7f3dac154230 2024-04-28T12:38:26.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.912+0000 7f3dab200640 1 -- 172.21.15.131:0/2274111420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dac1e5db0 con 0x7f3dac154230 2024-04-28T12:38:26.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.914+0000 7f3dab200640 1 -- 172.21.15.131:0/2274111420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dac0506c0 con 0x7f3dac154230 2024-04-28T12:38:26.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.915+0000 7f3da2a00640 1 -- 172.21.15.131:0/2274111420 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3da4042020 con 0x7f3dac154230 2024-04-28T12:38:26.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.916+0000 7f3da2a00640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3d8806c920 0x7f3d8806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:26.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.916+0000 7f3da2a00640 1 -- 172.21.15.131:0/2274111420 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3da40ab620 con 0x7f3dac154230 2024-04-28T12:38:26.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.916+0000 7f3da9e00640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3d8806c920 0x7f3d8806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:26.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.917+0000 7f3da9e00640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3d8806c920 0x7f3d8806ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c006fd0 tx=0x7f3d9c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:26.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:26.920+0000 7f3da2a00640 1 -- 172.21.15.131:0/2274111420 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3dac1e5570 con 0x7f3dac154230 2024-04-28T12:38:27.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.125+0000 7f3dab200640 1 -- 172.21.15.131:0/2274111420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3dac15e230 con 0x7f3dac154230 2024-04-28T12:38:27.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.126+0000 7f3da2a00640 1 -- 172.21.15.131:0/2274111420 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f3da4079630 con 0x7f3dac154230 2024-04-28T12:38:27.123 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:27.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.128+0000 7f3dab200640 1 -- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3d8806c920 msgr2=0x7f3d8806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.128+0000 7f3dab200640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3d8806c920 0x7f3d8806ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c006fd0 tx=0x7f3d9c008040 comp rx=0 tx=0).stop 2024-04-28T12:38:27.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.128+0000 7f3dab200640 1 -- 172.21.15.131:0/2274111420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac154230 msgr2=0x7f3dac157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.128+0000 7f3dab200640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac154230 0x7f3dac157940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3da4008fd0 tx=0x7f3da40014a0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.129+0000 7f3dab200640 1 -- 172.21.15.131:0/2274111420 shutdown_connections 2024-04-28T12:38:27.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.129+0000 7f3dab200640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3d8806c920 0x7f3d8806ede0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.129+0000 7f3dab200640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dac154230 0x7f3dac157940 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.129+0000 7f3dab200640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dac1e50f0 0x7f3dac157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.129+0000 7f3dab200640 1 --2- 172.21.15.131:0/2274111420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d90004d90 0x7f3dac1e6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.129+0000 7f3dab200640 1 -- 172.21.15.131:0/2274111420 >> 172.21.15.131:0/2274111420 conn(0x7f3dac068090 msgr2=0x7f3dac15ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:27.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.129+0000 7f3dab200640 1 -- 172.21.15.131:0/2274111420 shutdown_connections 2024-04-28T12:38:27.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.129+0000 7f3dab200640 1 -- 172.21.15.131:0/2274111420 wait complete. 2024-04-28T12:38:27.144 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:27.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.342+0000 7f93a5c00640 1 Processor -- start 2024-04-28T12:38:27.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.342+0000 7f93a5c00640 1 -- start start 2024-04-28T12:38:27.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.344+0000 7f93a5c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f93a00b5410 0x7f93a00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.344+0000 7f93a5c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f93a015d2b0 0x7f93a015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.344+0000 7f93a5c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93a00b3900 con 0x7f93a015a7e0 2024-04-28T12:38:27.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.344+0000 7f93a5c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f93a015abc0 con 0x7f93a00b5410 2024-04-28T12:38:27.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.344+0000 7f93a5c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f93a00b3b70 con 0x7f93a015d2b0 2024-04-28T12:38:27.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.344+0000 7f939fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f93a015a7e0 0x7f93a00b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43036/0 (socket says 172.21.15.131:43036) 2024-04-28T12:38:27.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.344+0000 7f939fe00640 1 -- 172.21.15.131:0/2369577196 learned_addr learned my addr 172.21.15.131:0/2369577196 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:27.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.344+0000 7f9397400640 1 --2- 172.21.15.131:0/2369577196 >> v2:172.21.15.131:3300/0 conn(0x7f93a00b5410 0x7f93a00b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.344+0000 7f93a5200640 1 --2- 172.21.15.131:0/2369577196 >> v2:172.21.15.157:3300/0 conn(0x7f93a015d2b0 0x7f93a015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.344+0000 7f939f400640 1 -- 172.21.15.131:0/2369577196 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 775820021 0 0) 0x7f93a00b3900 con 0x7f93a015a7e0 2024-04-28T12:38:27.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.344+0000 7f939f400640 1 -- 172.21.15.131:0/2369577196 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9384003520 con 0x7f93a015a7e0 2024-04-28T12:38:27.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.345+0000 7f9397400640 1 -- 172.21.15.131:0/2369577196 >> v2:172.21.15.157:3300/0 conn(0x7f93a015d2b0 msgr2=0x7f93a015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:27.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.345+0000 7f9397400640 1 --2- 172.21.15.131:0/2369577196 >> v2:172.21.15.157:3300/0 conn(0x7f93a015d2b0 0x7f93a015d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.345+0000 7f9397400640 1 -- 172.21.15.131:0/2369577196 >> v1:172.21.15.131:6789/0 conn(0x7f93a015a7e0 legacy=0x7f93a00b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.345+0000 7f9397400640 1 -- 172.21.15.131:0/2369577196 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a00b3e50 con 0x7f93a00b5410 2024-04-28T12:38:27.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.346+0000 7f9397400640 1 --2- 172.21.15.131:0/2369577196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93a00b5410 0x7f93a00b5890 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f93a00cd1c0 tx=0x7f938802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6876979961dd1245 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:27.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.346+0000 7f939f400640 1 -- 172.21.15.131:0/2369577196 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388037070 con 0x7f93a00b5410 2024-04-28T12:38:27.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.346+0000 7f939f400640 1 -- 172.21.15.131:0/2369577196 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9388032440 con 0x7f93a00b5410 2024-04-28T12:38:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.346+0000 7f939f400640 1 -- 172.21.15.131:0/2369577196 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388032810 con 0x7f93a00b5410 2024-04-28T12:38:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.346+0000 7f93a5c00640 1 -- 172.21.15.131:0/2369577196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93a00b5410 msgr2=0x7f93a00b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.346+0000 7f93a5c00640 1 --2- 172.21.15.131:0/2369577196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93a00b5410 0x7f93a00b5890 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f93a00cd1c0 tx=0x7f938802a1a0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.347+0000 7f93a5c00640 1 -- 172.21.15.131:0/2369577196 shutdown_connections 2024-04-28T12:38:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.347+0000 7f93a5c00640 1 --2- 172.21.15.131:0/2369577196 >> v2:172.21.15.157:3300/0 conn(0x7f93a015d2b0 0x7f93a015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.347+0000 7f93a5c00640 1 --2- 172.21.15.131:0/2369577196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93a00b5410 0x7f93a00b5890 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.347+0000 7f93a5c00640 1 -- 172.21.15.131:0/2369577196 >> 172.21.15.131:0/2369577196 conn(0x7f93a0068090 msgr2=0x7f93a00684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.347+0000 7f93a5c00640 1 -- 172.21.15.131:0/2369577196 shutdown_connections 2024-04-28T12:38:27.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.347+0000 7f93a5c00640 1 -- 172.21.15.131:0/2369577196 wait complete. 2024-04-28T12:38:27.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.348+0000 7f93a5c00640 1 Processor -- start 2024-04-28T12:38:27.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.349+0000 7f93a5c00640 1 -- start start 2024-04-28T12:38:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93a00b5410 0x7f93a0149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93a015a7e0 0x7f93a00cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93a015d2b0 0x7f93a00d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f93a015dc80 con 0x7f93a015d2b0 2024-04-28T12:38:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93a015e010 con 0x7f93a015a7e0 2024-04-28T12:38:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93a015e2e0 con 0x7f93a00b5410 2024-04-28T12:38:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93a015d2b0 0x7f93a00d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93a015d2b0 0x7f93a00d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54248/0 (socket says 172.21.15.131:54248) 2024-04-28T12:38:27.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5200640 1 -- 172.21.15.131:0/821986806 learned_addr learned my addr 172.21.15.131:0/821986806 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f939fe00640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93a00b5410 0x7f93a0149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f9397400640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93a015a7e0 0x7f93a00cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5200640 1 -- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93a00b5410 msgr2=0x7f93a0149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5200640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93a00b5410 0x7f93a0149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5200640 1 -- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93a015a7e0 msgr2=0x7f93a00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.350+0000 7f93a5200640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93a015a7e0 0x7f93a00cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.351+0000 7f93a5200640 1 -- 172.21.15.131:0/821986806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a0156070 con 0x7f93a015d2b0 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.351+0000 7f9397400640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93a015a7e0 0x7f93a00cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.351+0000 7f93a5200640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93a015d2b0 0x7f93a00d00b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9398007e60 tx=0x7f939800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.351+0000 7f939cc00640 1 -- 172.21.15.131:0/821986806 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398014420 con 0x7f93a015d2b0 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.351+0000 7f939cc00640 1 -- 172.21.15.131:0/821986806 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9398019070 con 0x7f93a015d2b0 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.351+0000 7f939cc00640 1 -- 172.21.15.131:0/821986806 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398014820 con 0x7f93a015d2b0 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.351+0000 7f93a5c00640 1 -- 172.21.15.131:0/821986806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a0156350 con 0x7f93a015d2b0 2024-04-28T12:38:27.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.351+0000 7f93a5c00640 1 -- 172.21.15.131:0/821986806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a0156910 con 0x7f93a015d2b0 2024-04-28T12:38:27.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.352+0000 7f93a5c00640 1 -- 172.21.15.131:0/821986806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a00506c0 con 0x7f93a015d2b0 2024-04-28T12:38:27.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.354+0000 7f939cc00640 1 -- 172.21.15.131:0/821986806 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93980149c0 con 0x7f93a015d2b0 2024-04-28T12:38:27.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.354+0000 7f939cc00640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f937c06c920 0x7f937c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.355+0000 7f939cc00640 1 -- 172.21.15.131:0/821986806 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f939808c7e0 con 0x7f93a015d2b0 2024-04-28T12:38:27.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.355+0000 7f939fe00640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f937c06c920 0x7f937c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.356+0000 7f939fe00640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f937c06c920 0x7f937c06ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9390005880 tx=0x7f9390009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:27.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.359+0000 7f939cc00640 1 -- 172.21.15.131:0/821986806 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f939805a800 con 0x7f93a015d2b0 2024-04-28T12:38:27.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.557+0000 7f93a5c00640 1 -- 172.21.15.131:0/821986806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f93a00af8d0 con 0x7f93a015d2b0 2024-04-28T12:38:27.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.558+0000 7f939cc00640 1 -- 172.21.15.131:0/821986806 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f939805a100 con 0x7f93a015d2b0 2024-04-28T12:38:27.555 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:27.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.561+0000 7f93a5c00640 1 -- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f937c06c920 msgr2=0x7f937c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.561+0000 7f93a5c00640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f937c06c920 0x7f937c06ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9390005880 tx=0x7f9390009040 comp rx=0 tx=0).stop 2024-04-28T12:38:27.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.561+0000 7f93a5c00640 1 -- 172.21.15.131:0/821986806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93a015d2b0 msgr2=0x7f93a00d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.561+0000 7f93a5c00640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93a015d2b0 0x7f93a00d00b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9398007e60 tx=0x7f939800e4b0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.561+0000 7f93a5c00640 1 -- 172.21.15.131:0/821986806 shutdown_connections 2024-04-28T12:38:27.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.561+0000 7f93a5c00640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f937c06c920 0x7f937c06ede0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.561+0000 7f93a5c00640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93a015d2b0 0x7f93a00d00b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.561+0000 7f93a5c00640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93a015a7e0 0x7f93a00cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.561+0000 7f93a5c00640 1 --2- 172.21.15.131:0/821986806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93a00b5410 0x7f93a0149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.561+0000 7f93a5c00640 1 -- 172.21.15.131:0/821986806 >> 172.21.15.131:0/821986806 conn(0x7f93a0068090 msgr2=0x7f93a0068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:27.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.561+0000 7f93a5c00640 1 -- 172.21.15.131:0/821986806 shutdown_connections 2024-04-28T12:38:27.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.561+0000 7f93a5c00640 1 -- 172.21.15.131:0/821986806 wait complete. 2024-04-28T12:38:27.576 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.784+0000 7f655ac00640 1 Processor -- start 2024-04-28T12:38:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.784+0000 7f655ac00640 1 -- start start 2024-04-28T12:38:27.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.786+0000 7f655ac00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6554156100 0x7f65540d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.786+0000 7f655ac00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65540b57c0 con 0x7f65540b5200 2024-04-28T12:38:27.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.786+0000 7f655ac00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f655415c230 con 0x7f65540d1110 2024-04-28T12:38:27.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.786+0000 7f655ac00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f65540d1560 con 0x7f6554156100 2024-04-28T12:38:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.786+0000 7f655a200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f65540b5200 0x7f65540b3c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43042/0 (socket says 172.21.15.131:43042) 2024-04-28T12:38:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.786+0000 7f655a200640 1 -- 172.21.15.131:0/4251950661 learned_addr learned my addr 172.21.15.131:0/4251950661 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6559800640 1 --2- 172.21.15.131:0/4251950661 >> v2:172.21.15.157:3300/0 conn(0x7f6554156100 0x7f65540d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3847989366 0 0) 0x7f654c001490 con 0x7f65540b5200 2024-04-28T12:38:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f653c005730 con 0x7f65540b5200 2024-04-28T12:38:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1998752251 0 0) 0x7f653c005730 con 0x7f65540b5200 2024-04-28T12:38:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 >> v2:172.21.15.157:3300/0 conn(0x7f6554156100 msgr2=0x7f65540d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6553e00640 1 --2- 172.21.15.131:0/4251950661 >> v2:172.21.15.157:3300/0 conn(0x7f6554156100 0x7f65540d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 >> v1:172.21.15.157:6789/0 conn(0x7f65540d1110 legacy=0x7f65540b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655415c510 con 0x7f65540b5200 2024-04-28T12:38:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f654c001bc0 con 0x7f65540b5200 2024-04-28T12:38:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6559800640 1 --2- 172.21.15.131:0/4251950661 >> v2:172.21.15.157:3300/0 conn(0x7f6554156100 0x7f65540d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:38:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 >> v1:172.21.15.131:6789/0 conn(0x7f65540b5200 legacy=0x7f65540b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6553e00640 1 --2- 172.21.15.131:0/4251950661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f653c005c30 0x7f653c006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.787+0000 7f6553e00640 1 --2- 172.21.15.131:0/4251950661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f653c006a10 0x7f653c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f6553e00640 1 --2- 172.21.15.131:0/4251950661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c0073d0 0x7f653c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f65540d1560 con 0x7f653c005c30 2024-04-28T12:38:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f653c00e300 con 0x7f653c006a10 2024-04-28T12:38:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f653c00e500 con 0x7f653c0073d0 2024-04-28T12:38:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f6559800640 1 --2- 172.21.15.131:0/4251950661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f653c005c30 0x7f653c006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f6558e00640 1 --2- 172.21.15.131:0/4251950661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f653c006a10 0x7f653c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f655a200640 1 --2- 172.21.15.131:0/4251950661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c0073d0 0x7f653c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f6559800640 1 -- 172.21.15.131:0/4251950661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c0073d0 msgr2=0x7f653c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:27.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f6559800640 1 --2- 172.21.15.131:0/4251950661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c0073d0 0x7f653c00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f6559800640 1 -- 172.21.15.131:0/4251950661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f653c006a10 msgr2=0x7f653c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:27.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f6559800640 1 --2- 172.21.15.131:0/4251950661 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f653c006a10 0x7f653c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f6559800640 1 -- 172.21.15.131:0/4251950661 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653c005730 con 0x7f653c005c30 2024-04-28T12:38:27.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f655a200640 1 --2- 172.21.15.131:0/4251950661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c0073d0 0x7f653c00dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:27.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.788+0000 7f655a200640 1 -- 172.21.15.131:0/4251950661 reap_dead start 2024-04-28T12:38:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.789+0000 7f6559800640 1 --2- 172.21.15.131:0/4251950661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f653c005c30 0x7f653c006050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6548009c90 tx=0x7f653c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93870b553fd86a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.790+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548036070 con 0x7f653c005c30 2024-04-28T12:38:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.790+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6548031400 con 0x7f653c005c30 2024-04-28T12:38:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.790+0000 7f6553e00640 1 -- 172.21.15.131:0/4251950661 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548009860 con 0x7f653c005c30 2024-04-28T12:38:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.790+0000 7f655ac00640 1 -- 172.21.15.131:0/4251950661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f653c005c30 msgr2=0x7f653c006050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.790+0000 7f655ac00640 1 --2- 172.21.15.131:0/4251950661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f653c005c30 0x7f653c006050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6548009c90 tx=0x7f653c007a40 comp rx=0 tx=0).stop 2024-04-28T12:38:27.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.790+0000 7f655ac00640 1 -- 172.21.15.131:0/4251950661 shutdown_connections 2024-04-28T12:38:27.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.790+0000 7f655ac00640 1 --2- 172.21.15.131:0/4251950661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f653c005c30 0x7f653c006050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.790+0000 7f655ac00640 1 -- 172.21.15.131:0/4251950661 >> 172.21.15.131:0/4251950661 conn(0x7f65540684e0 msgr2=0x7f65540688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:27.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.790+0000 7f655ac00640 1 -- 172.21.15.131:0/4251950661 shutdown_connections 2024-04-28T12:38:27.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.790+0000 7f655ac00640 1 -- 172.21.15.131:0/4251950661 wait complete. 2024-04-28T12:38:27.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.792+0000 7f655ac00640 1 Processor -- start 2024-04-28T12:38:27.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.792+0000 7f655ac00640 1 -- start start 2024-04-28T12:38:27.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.793+0000 7f655ac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c005c30 0x7f65541ea920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.793+0000 7f655ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65540ce760 0x7f655415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.793+0000 7f655ac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655415f860 0x7f65541f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.793+0000 7f655ac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f65540b3860 con 0x7f655415f860 2024-04-28T12:38:27.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.793+0000 7f655ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f655415cad0 con 0x7f65540ce760 2024-04-28T12:38:27.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.793+0000 7f655ac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65540b39e0 con 0x7f653c005c30 2024-04-28T12:38:27.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.794+0000 7f655a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655415f860 0x7f65541f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.794+0000 7f655a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655415f860 0x7f65541f55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54262/0 (socket says 172.21.15.131:54262) 2024-04-28T12:38:27.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.794+0000 7f655a200640 1 -- 172.21.15.131:0/3926050382 learned_addr learned my addr 172.21.15.131:0/3926050382 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:27.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.794+0000 7f6559800640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c005c30 0x7f65541ea920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.794+0000 7f6558e00640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65540ce760 0x7f655415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.794+0000 7f655a200640 1 -- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c005c30 msgr2=0x7f65541ea920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.794+0000 7f655a200640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c005c30 0x7f65541ea920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.794+0000 7f655a200640 1 -- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65540ce760 msgr2=0x7f655415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:27.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.794+0000 7f655a200640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65540ce760 0x7f655415f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:27.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.794+0000 7f655a200640 1 -- 172.21.15.131:0/3926050382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65541f6310 con 0x7f655415f860 2024-04-28T12:38:27.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.794+0000 7f655a200640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655415f860 0x7f65541f55e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f654c0012e0 tx=0x7f654c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:27.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.795+0000 7f6551600640 1 -- 172.21.15.131:0/3926050382 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c001bc0 con 0x7f655415f860 2024-04-28T12:38:27.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.795+0000 7f6551600640 1 -- 172.21.15.131:0/3926050382 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f654c00bb20 con 0x7f655415f860 2024-04-28T12:38:27.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.795+0000 7f655ac00640 1 -- 172.21.15.131:0/3926050382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65540cdc80 con 0x7f655415f860 2024-04-28T12:38:27.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.795+0000 7f6551600640 1 -- 172.21.15.131:0/3926050382 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c007070 con 0x7f655415f860 2024-04-28T12:38:27.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.795+0000 7f655ac00640 1 -- 172.21.15.131:0/3926050382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65540ce240 con 0x7f655415f860 2024-04-28T12:38:27.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.796+0000 7f655ac00640 1 -- 172.21.15.131:0/3926050382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65540506c0 con 0x7f655415f860 2024-04-28T12:38:27.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.797+0000 7f6551600640 1 -- 172.21.15.131:0/3926050382 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f654c01b060 con 0x7f655415f860 2024-04-28T12:38:27.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.798+0000 7f6551600640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f653406c920 0x7f653406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:27.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.798+0000 7f6551600640 1 -- 172.21.15.131:0/3926050382 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f654c08bf30 con 0x7f655415f860 2024-04-28T12:38:27.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.798+0000 7f6559800640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f653406c920 0x7f653406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:27.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.800+0000 7f6559800640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f653406c920 0x7f653406ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6548033000 tx=0x7f6548008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:27.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:27.802+0000 7f6551600640 1 -- 172.21.15.131:0/3926050382 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f65540b57c0 con 0x7f655415f860 2024-04-28T12:38:27.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.002+0000 7f655ac00640 1 -- 172.21.15.131:0/3926050382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f65540aff40 con 0x7f655415f860 2024-04-28T12:38:28.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.003+0000 7f6551600640 1 -- 172.21.15.131:0/3926050382 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f654c059ec0 con 0x7f655415f860 2024-04-28T12:38:28.000 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:28.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.006+0000 7f655ac00640 1 -- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f653406c920 msgr2=0x7f653406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:28.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.006+0000 7f655ac00640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f653406c920 0x7f653406ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6548033000 tx=0x7f6548008030 comp rx=0 tx=0).stop 2024-04-28T12:38:28.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.006+0000 7f655ac00640 1 -- 172.21.15.131:0/3926050382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655415f860 msgr2=0x7f65541f55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:28.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.006+0000 7f655ac00640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655415f860 0x7f65541f55e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f654c0012e0 tx=0x7f654c00b4a0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.006+0000 7f655ac00640 1 -- 172.21.15.131:0/3926050382 shutdown_connections 2024-04-28T12:38:28.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.006+0000 7f655ac00640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f653406c920 0x7f653406ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.006+0000 7f655ac00640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f655415f860 0x7f65541f55e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.006+0000 7f655ac00640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65540ce760 0x7f655415f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.006+0000 7f655ac00640 1 --2- 172.21.15.131:0/3926050382 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f653c005c30 0x7f65541ea920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.006+0000 7f655ac00640 1 -- 172.21.15.131:0/3926050382 >> 172.21.15.131:0/3926050382 conn(0x7f65540684e0 msgr2=0x7f655415b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:28.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.006+0000 7f655ac00640 1 -- 172.21.15.131:0/3926050382 shutdown_connections 2024-04-28T12:38:28.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.006+0000 7f655ac00640 1 -- 172.21.15.131:0/3926050382 wait complete. 2024-04-28T12:38:28.022 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.234+0000 7f79b1200640 1 Processor -- start 2024-04-28T12:38:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.234+0000 7f79b1200640 1 -- start start 2024-04-28T12:38:28.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.236+0000 7f79b1200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f79ac15f180 0x7f79ac0d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.236+0000 7f79b1200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79ac0d2d70 con 0x7f79ac0b3cb0 2024-04-28T12:38:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.236+0000 7f79b1200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79ac0d06e0 con 0x7f79ac0d3830 2024-04-28T12:38:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.236+0000 7f79b1200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f79ac15f560 con 0x7f79ac15f180 2024-04-28T12:38:28.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.236+0000 7f79a3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f79ac0b3cb0 0x7f79ac0d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43048/0 (socket says 172.21.15.131:43048) 2024-04-28T12:38:28.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.236+0000 7f79a3e00640 1 -- 172.21.15.131:0/155068711 learned_addr learned my addr 172.21.15.131:0/155068711 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.237+0000 7f79ab400640 1 --2- 172.21.15.131:0/155068711 >> v2:172.21.15.157:3300/0 conn(0x7f79ac15f180 0x7f79ac0d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.237+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 741346664 0 0) 0x7f79ac0d2d70 con 0x7f79ac0b3cb0 2024-04-28T12:38:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.237+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7990003520 con 0x7f79ac0b3cb0 2024-04-28T12:38:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.237+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3197747736 0 0) 0x7f7990003520 con 0x7f79ac0b3cb0 2024-04-28T12:38:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.237+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 >> v2:172.21.15.157:3300/0 conn(0x7f79ac15f180 msgr2=0x7f79ac0d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.237+0000 7f79aaa00640 1 --2- 172.21.15.131:0/155068711 >> v2:172.21.15.157:3300/0 conn(0x7f79ac15f180 0x7f79ac0d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.237+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 >> v1:172.21.15.157:6789/0 conn(0x7f79ac0d3830 legacy=0x7f79ac0b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.237+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ac0d09c0 con 0x7f79ac0b3cb0 2024-04-28T12:38:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.237+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f7994001690 con 0x7f79ac0b3cb0 2024-04-28T12:38:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.237+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 >> v1:172.21.15.131:6789/0 conn(0x7f79ac0b3cb0 legacy=0x7f79ac0d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.237+0000 7f79aaa00640 1 --2- 172.21.15.131:0/155068711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 0x7f7990003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79aaa00640 1 --2- 172.21.15.131:0/155068711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7990004800 0x7f7990004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79aaa00640 1 --2- 172.21.15.131:0/155068711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79900051c0 0x7f799000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac15f560 con 0x7f7990003a20 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f799000c0f0 con 0x7f79900051c0 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f799000c2f0 con 0x7f7990004800 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79ab400640 1 --2- 172.21.15.131:0/155068711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 0x7f7990003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79a3e00640 1 --2- 172.21.15.131:0/155068711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7990004800 0x7f7990004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79ab400640 1 -- 172.21.15.131:0/155068711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7990004800 msgr2=0x7f7990004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79abe00640 1 --2- 172.21.15.131:0/155068711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79900051c0 0x7f799000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79ab400640 1 --2- 172.21.15.131:0/155068711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7990004800 0x7f7990004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79ab400640 1 -- 172.21.15.131:0/155068711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79900051c0 msgr2=0x7f799000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79ab400640 1 --2- 172.21.15.131:0/155068711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79900051c0 0x7f799000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79ab400640 1 -- 172.21.15.131:0/155068711 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7990003520 con 0x7f7990003a20 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79abe00640 1 --2- 172.21.15.131:0/155068711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79900051c0 0x7f799000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.238+0000 7f79abe00640 1 -- 172.21.15.131:0/155068711 reap_dead start 2024-04-28T12:38:28.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.239+0000 7f79ab400640 1 --2- 172.21.15.131:0/155068711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 0x7f7990003e40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f799c009ac0 tx=0x7f7990005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9facfd64adae5d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:28.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.240+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c036070 con 0x7f7990003a20 2024-04-28T12:38:28.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.240+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799c031400 con 0x7f7990003a20 2024-04-28T12:38:28.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.240+0000 7f79aaa00640 1 -- 172.21.15.131:0/155068711 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c009690 con 0x7f7990003a20 2024-04-28T12:38:28.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.240+0000 7f79b1200640 1 -- 172.21.15.131:0/155068711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 msgr2=0x7f7990003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:28.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.240+0000 7f79b1200640 1 --2- 172.21.15.131:0/155068711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 0x7f7990003e40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f799c009ac0 tx=0x7f7990005830 comp rx=0 tx=0).stop 2024-04-28T12:38:28.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.240+0000 7f79b1200640 1 -- 172.21.15.131:0/155068711 shutdown_connections 2024-04-28T12:38:28.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.240+0000 7f79b1200640 1 --2- 172.21.15.131:0/155068711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 0x7f7990003e40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.240+0000 7f79b1200640 1 -- 172.21.15.131:0/155068711 >> 172.21.15.131:0/155068711 conn(0x7f79ac0d4bd0 msgr2=0x7f79ac0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:28.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.240+0000 7f79b1200640 1 -- 172.21.15.131:0/155068711 shutdown_connections 2024-04-28T12:38:28.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.240+0000 7f79b1200640 1 -- 172.21.15.131:0/155068711 wait complete. 2024-04-28T12:38:28.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.242+0000 7f79b1200640 1 Processor -- start 2024-04-28T12:38:28.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.242+0000 7f79b1200640 1 -- start start 2024-04-28T12:38:28.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.243+0000 7f79b1200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 0x7f79ac156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.243+0000 7f79b1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ac0c8700 0x7f79ac15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.243+0000 7f79b1200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79ac0c8b80 0x7f79ac15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.243+0000 7f79b1200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac0d11c0 con 0x7f7990003a20 2024-04-28T12:38:28.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.243+0000 7f79b1200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac0d2a40 con 0x7f79ac0c8700 2024-04-28T12:38:28.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.243+0000 7f79b1200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f79ac0c9150 con 0x7f79ac0c8b80 2024-04-28T12:38:28.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.243+0000 7f79ab400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 0x7f79ac156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:28.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.243+0000 7f79ab400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 0x7f79ac156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54274/0 (socket says 172.21.15.131:54274) 2024-04-28T12:38:28.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.244+0000 7f79ab400640 1 -- 172.21.15.131:0/3300844713 learned_addr learned my addr 172.21.15.131:0/3300844713 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.244+0000 7f79ab400640 1 -- 172.21.15.131:0/3300844713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79ac0c8b80 msgr2=0x7f79ac15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.244+0000 7f79ab400640 1 --2- 172.21.15.131:0/3300844713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79ac0c8b80 0x7f79ac15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.244+0000 7f79ab400640 1 -- 172.21.15.131:0/3300844713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ac0c8700 msgr2=0x7f79ac15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.244+0000 7f79ab400640 1 --2- 172.21.15.131:0/3300844713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ac0c8700 0x7f79ac15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.244+0000 7f79ab400640 1 -- 172.21.15.131:0/3300844713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ac0c9430 con 0x7f7990003a20 2024-04-28T12:38:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.244+0000 7f79ab400640 1 --2- 172.21.15.131:0/3300844713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 0x7f79ac156430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f799c034040 tx=0x7f799c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.244+0000 7f79a3400640 1 -- 172.21.15.131:0/3300844713 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c03f020 con 0x7f7990003a20 2024-04-28T12:38:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.245+0000 7f79a3400640 1 -- 172.21.15.131:0/3300844713 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799c037040 con 0x7f7990003a20 2024-04-28T12:38:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.245+0000 7f79a3400640 1 -- 172.21.15.131:0/3300844713 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c036070 con 0x7f7990003a20 2024-04-28T12:38:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.245+0000 7f79b1200640 1 -- 172.21.15.131:0/3300844713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ac0c9710 con 0x7f7990003a20 2024-04-28T12:38:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.245+0000 7f79b1200640 1 -- 172.21.15.131:0/3300844713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ac0b7e00 con 0x7f7990003a20 2024-04-28T12:38:28.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.246+0000 7f79b1200640 1 -- 172.21.15.131:0/3300844713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79ac0506c0 con 0x7f7990003a20 2024-04-28T12:38:28.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.247+0000 7f79a3400640 1 -- 172.21.15.131:0/3300844713 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f799c042020 con 0x7f7990003a20 2024-04-28T12:38:28.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.248+0000 7f79a3400640 1 --2- 172.21.15.131:0/3300844713 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f798806c920 0x7f798806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.248+0000 7f79a3400640 1 -- 172.21.15.131:0/3300844713 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f799c0abff0 con 0x7f7990003a20 2024-04-28T12:38:28.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.248+0000 7f79a3e00640 1 --2- 172.21.15.131:0/3300844713 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f798806c920 0x7f798806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:28.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.250+0000 7f79a3e00640 1 --2- 172.21.15.131:0/3300844713 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f798806c920 0x7f798806ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7994001800 tx=0x7f7994009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:28.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.252+0000 7f79a3400640 1 -- 172.21.15.131:0/3300844713 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f799c049030 con 0x7f7990003a20 2024-04-28T12:38:28.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.451+0000 7f79b1200640 1 -- 172.21.15.131:0/3300844713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f79ac0b7460 con 0x7f7990003a20 2024-04-28T12:38:28.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.452+0000 7f79a3400640 1 -- 172.21.15.131:0/3300844713 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f799c07a040 con 0x7f7990003a20 2024-04-28T12:38:28.448 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.454+0000 7f79b1200640 1 -- 172.21.15.131:0/3300844713 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f798806c920 msgr2=0x7f798806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.454+0000 7f79b1200640 1 --2- 172.21.15.131:0/3300844713 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f798806c920 0x7f798806ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7994001800 tx=0x7f7994009040 comp rx=0 tx=0).stop 2024-04-28T12:38:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.454+0000 7f79b1200640 1 -- 172.21.15.131:0/3300844713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 msgr2=0x7f79ac156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.454+0000 7f79b1200640 1 --2- 172.21.15.131:0/3300844713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 0x7f79ac156430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f799c034040 tx=0x7f799c004730 comp rx=0 tx=0).stop 2024-04-28T12:38:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.454+0000 7f79b1200640 1 -- 172.21.15.131:0/3300844713 shutdown_connections 2024-04-28T12:38:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.454+0000 7f79b1200640 1 --2- 172.21.15.131:0/3300844713 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f798806c920 0x7f798806ede0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.454+0000 7f79b1200640 1 --2- 172.21.15.131:0/3300844713 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f79ac0c8b80 0x7f79ac15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.454+0000 7f79b1200640 1 --2- 172.21.15.131:0/3300844713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79ac0c8700 0x7f79ac15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.454+0000 7f79b1200640 1 --2- 172.21.15.131:0/3300844713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7990003a20 0x7f79ac156430 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.454+0000 7f79b1200640 1 -- 172.21.15.131:0/3300844713 >> 172.21.15.131:0/3300844713 conn(0x7f79ac0d4bd0 msgr2=0x7f79ac15c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.455+0000 7f79b1200640 1 -- 172.21.15.131:0/3300844713 shutdown_connections 2024-04-28T12:38:28.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.455+0000 7f79b1200640 1 -- 172.21.15.131:0/3300844713 wait complete. 2024-04-28T12:38:28.470 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-04-28T12:38:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.672+0000 7ff092400640 1 Processor -- start 2024-04-28T12:38:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.672+0000 7ff092400640 1 -- start start 2024-04-28T12:38:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.674+0000 7ff092400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff08c0b5c10 0x7ff08c072500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.674+0000 7ff092400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08c1600b0 0x7ff08c072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.674+0000 7ff092400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff08c0b3c90 con 0x7ff08c15fcd0 2024-04-28T12:38:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.674+0000 7ff092400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff08c0b4b20 con 0x7ff08c0b5c10 2024-04-28T12:38:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.674+0000 7ff092400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff08c0af8d0 con 0x7ff08c1600b0 2024-04-28T12:38:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.674+0000 7ff083e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff08c0b5c10 0x7ff08c072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.674+0000 7ff083e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff08c0b5c10 0x7ff08c072500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54282/0 (socket says 172.21.15.131:54282) 2024-04-28T12:38:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.674+0000 7ff083e00640 1 -- 172.21.15.131:0/912449125 learned_addr learned my addr 172.21.15.131:0/912449125 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:28.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.675+0000 7ff091a00640 1 --2- 172.21.15.131:0/912449125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08c1600b0 0x7ff08c072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:28.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.675+0000 7ff083e00640 1 -- 172.21.15.131:0/912449125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08c1600b0 msgr2=0x7ff08c072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:28.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.675+0000 7ff083e00640 1 --2- 172.21.15.131:0/912449125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08c1600b0 0x7ff08c072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.675+0000 7ff083e00640 1 -- 172.21.15.131:0/912449125 >> v1:172.21.15.157:6789/0 conn(0x7ff08c15fcd0 legacy=0x7ff08c0b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:28.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.675+0000 7ff083e00640 1 -- 172.21.15.131:0/912449125 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c0b3f10 con 0x7ff08c0b5c10 2024-04-28T12:38:28.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.676+0000 7ff083e00640 1 --2- 172.21.15.131:0/912449125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c0b5c10 0x7ff08c072500 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff074009870 tx=0x7ff07402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881e51e9dcb08129 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:28.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.676+0000 7ff08be00640 1 -- 172.21.15.131:0/912449125 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074037070 con 0x7ff08c0b5c10 2024-04-28T12:38:28.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.676+0000 7ff08be00640 1 -- 172.21.15.131:0/912449125 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff074032440 con 0x7ff08c0b5c10 2024-04-28T12:38:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.676+0000 7ff08be00640 1 -- 172.21.15.131:0/912449125 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074032810 con 0x7ff08c0b5c10 2024-04-28T12:38:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.677+0000 7ff092400640 1 -- 172.21.15.131:0/912449125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c0b5c10 msgr2=0x7ff08c072500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.677+0000 7ff092400640 1 --2- 172.21.15.131:0/912449125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c0b5c10 0x7ff08c072500 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff074009870 tx=0x7ff07402a210 comp rx=0 tx=0).stop 2024-04-28T12:38:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.677+0000 7ff092400640 1 -- 172.21.15.131:0/912449125 shutdown_connections 2024-04-28T12:38:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.677+0000 7ff092400640 1 --2- 172.21.15.131:0/912449125 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08c1600b0 0x7ff08c072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.677+0000 7ff092400640 1 --2- 172.21.15.131:0/912449125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c0b5c10 0x7ff08c072500 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.677+0000 7ff092400640 1 -- 172.21.15.131:0/912449125 >> 172.21.15.131:0/912449125 conn(0x7ff08c0b26b0 msgr2=0x7ff08c0b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.677+0000 7ff092400640 1 -- 172.21.15.131:0/912449125 shutdown_connections 2024-04-28T12:38:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.677+0000 7ff092400640 1 -- 172.21.15.131:0/912449125 wait complete. 2024-04-28T12:38:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.679+0000 7ff092400640 1 Processor -- start 2024-04-28T12:38:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.679+0000 7ff092400640 1 -- start start 2024-04-28T12:38:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.680+0000 7ff092400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c0b5c10 0x7ff08c14e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.680+0000 7ff092400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08c15fcd0 0x7ff08c14ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.680+0000 7ff092400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff08c1600b0 0x7ff08c14d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.680+0000 7ff092400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c0b47b0 con 0x7ff08c0b5c10 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.680+0000 7ff092400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c0b44d0 con 0x7ff08c1600b0 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.680+0000 7ff092400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff08c0b4fb0 con 0x7ff08c15fcd0 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.680+0000 7ff091000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c0b5c10 0x7ff08c14e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.680+0000 7ff091000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c0b5c10 0x7ff08c14e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54288/0 (socket says 172.21.15.131:54288) 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.680+0000 7ff091000640 1 -- 172.21.15.131:0/1178043420 learned_addr learned my addr 172.21.15.131:0/1178043420 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.681+0000 7ff083e00640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08c15fcd0 0x7ff08c14ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.681+0000 7ff091000640 1 -- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08c15fcd0 msgr2=0x7ff08c14ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.681+0000 7ff091000640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08c15fcd0 0x7ff08c14ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.681+0000 7ff091000640 1 -- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff08c1600b0 msgr2=0x7ff08c14d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.681+0000 7ff091000640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff08c1600b0 0x7ff08c14d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.681+0000 7ff091000640 1 -- 172.21.15.131:0/1178043420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c14c180 con 0x7ff08c0b5c10 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.681+0000 7ff083e00640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08c15fcd0 0x7ff08c14ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.681+0000 7ff091000640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c0b5c10 0x7ff08c14e5a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00c7f0 tx=0x7ff07c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.681+0000 7ff089600640 1 -- 172.21.15.131:0/1178043420 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c016020 con 0x7ff08c0b5c10 2024-04-28T12:38:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.681+0000 7ff089600640 1 -- 172.21.15.131:0/1178043420 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07c00aa50 con 0x7ff08c0b5c10 2024-04-28T12:38:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.681+0000 7ff089600640 1 -- 172.21.15.131:0/1178043420 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00ae40 con 0x7ff08c0b5c10 2024-04-28T12:38:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.681+0000 7ff092400640 1 -- 172.21.15.131:0/1178043420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c14c460 con 0x7ff08c0b5c10 2024-04-28T12:38:28.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.682+0000 7ff092400640 1 -- 172.21.15.131:0/1178043420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c14ca20 con 0x7ff08c0b5c10 2024-04-28T12:38:28.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.683+0000 7ff092400640 1 -- 172.21.15.131:0/1178043420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c069350 con 0x7ff08c0b5c10 2024-04-28T12:38:28.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.684+0000 7ff089600640 1 -- 172.21.15.131:0/1178043420 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff07c019020 con 0x7ff08c0b5c10 2024-04-28T12:38:28.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.685+0000 7ff089600640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff06806c8f0 0x7ff06806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:28.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.685+0000 7ff089600640 1 -- 172.21.15.131:0/1178043420 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff07c08bcf0 con 0x7ff08c0b5c10 2024-04-28T12:38:28.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.685+0000 7ff083e00640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff06806c8f0 0x7ff06806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.686+0000 7ff083e00640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff06806c8f0 0x7ff06806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff074034000 tx=0x7ff07402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.689+0000 7ff089600640 1 -- 172.21.15.131:0/1178043420 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff07c020070 con 0x7ff08c0b5c10 2024-04-28T12:38:28.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:28.966+0000 7ff092400640 1 -- 172.21.15.131:0/1178043420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) -- 0x7ff08c0b59c0 con 0x7ff08c0b5c10 2024-04-28T12:38:29.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.313+0000 7ff089600640 1 -- 172.21.15.131:0/1178043420 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v82) ==== 123+0+0 (secure 0 0 0) 0x7ff07c059d10 con 0x7ff08c0b5c10 2024-04-28T12:38:29.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.316+0000 7ff092400640 1 -- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff06806c8f0 msgr2=0x7ff06806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:29.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.316+0000 7ff092400640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff06806c8f0 0x7ff06806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff074034000 tx=0x7ff07402e000 comp rx=0 tx=0).stop 2024-04-28T12:38:29.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.316+0000 7ff092400640 1 -- 172.21.15.131:0/1178043420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c0b5c10 msgr2=0x7ff08c14e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:29.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.316+0000 7ff092400640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c0b5c10 0x7ff08c14e5a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00c7f0 tx=0x7ff07c00ccb0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.316+0000 7ff092400640 1 -- 172.21.15.131:0/1178043420 shutdown_connections 2024-04-28T12:38:29.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.316+0000 7ff092400640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff06806c8f0 0x7ff06806edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.316+0000 7ff092400640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff08c1600b0 0x7ff08c14d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.316+0000 7ff092400640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff08c15fcd0 0x7ff08c14ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.316+0000 7ff092400640 1 --2- 172.21.15.131:0/1178043420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff08c0b5c10 0x7ff08c14e5a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.316+0000 7ff092400640 1 -- 172.21.15.131:0/1178043420 >> 172.21.15.131:0/1178043420 conn(0x7ff08c0b26b0 msgr2=0x7ff08c0721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:29.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.316+0000 7ff092400640 1 -- 172.21.15.131:0/1178043420 shutdown_connections 2024-04-28T12:38:29.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.317+0000 7ff092400640 1 -- 172.21.15.131:0/1178043420 wait complete. 2024-04-28T12:38:29.336 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-28T12:38:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.540+0000 7fe755600640 1 Processor -- start 2024-04-28T12:38:29.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.540+0000 7fe755600640 1 -- start start 2024-04-28T12:38:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.541+0000 7fe755600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe75015b980 0x7fe75006ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.541+0000 7fe755600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe75015bd60 0x7fe75006f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.541+0000 7fe755600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe75015c2a0 con 0x7fe7500b4610 2024-04-28T12:38:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.542+0000 7fe755600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe75015d060 con 0x7fe75015bd60 2024-04-28T12:38:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.542+0000 7fe755600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe75015c580 con 0x7fe75015b980 2024-04-28T12:38:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.542+0000 7fe747e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe75015bd60 0x7fe75006f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:29.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.542+0000 7fe754c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe7500b4610 0x7fe7500b4aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43056/0 (socket says 172.21.15.131:43056) 2024-04-28T12:38:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.542+0000 7fe754c00640 1 -- 172.21.15.131:0/2660191807 learned_addr learned my addr 172.21.15.131:0/2660191807 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.542+0000 7fe74fe00640 1 --2- 172.21.15.131:0/2660191807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe75015b980 0x7fe75006ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.542+0000 7fe747e00640 1 -- 172.21.15.131:0/2660191807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe75015b980 msgr2=0x7fe75006ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.542+0000 7fe747e00640 1 --2- 172.21.15.131:0/2660191807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe75015b980 0x7fe75006ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.542+0000 7fe747e00640 1 -- 172.21.15.131:0/2660191807 >> v1:172.21.15.131:6789/0 conn(0x7fe7500b4610 legacy=0x7fe7500b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.542+0000 7fe747e00640 1 -- 172.21.15.131:0/2660191807 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75015c860 con 0x7fe75015bd60 2024-04-28T12:38:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.542+0000 7fe74f400640 1 -- 172.21.15.131:0/2660191807 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367971600 0 0) 0x7fe75015c2a0 con 0x7fe7500b4610 2024-04-28T12:38:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.542+0000 7fe74fe00640 1 --2- 172.21.15.131:0/2660191807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe75015b980 0x7fe75006ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.543+0000 7fe747e00640 1 --2- 172.21.15.131:0/2660191807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe75015bd60 0x7fe75006f1d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe7380097a0 tx=0x7fe73802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a613eb763367171 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.544+0000 7fe74f400640 1 -- 172.21.15.131:0/2660191807 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738037070 con 0x7fe75015bd60 2024-04-28T12:38:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.544+0000 7fe74f400640 1 -- 172.21.15.131:0/2660191807 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe738032440 con 0x7fe75015bd60 2024-04-28T12:38:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.544+0000 7fe74f400640 1 -- 172.21.15.131:0/2660191807 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738032810 con 0x7fe75015bd60 2024-04-28T12:38:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.544+0000 7fe755600640 1 -- 172.21.15.131:0/2660191807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe75015bd60 msgr2=0x7fe75006f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.544+0000 7fe755600640 1 --2- 172.21.15.131:0/2660191807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe75015bd60 0x7fe75006f1d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe7380097a0 tx=0x7fe73802a180 comp rx=0 tx=0).stop 2024-04-28T12:38:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.544+0000 7fe755600640 1 -- 172.21.15.131:0/2660191807 shutdown_connections 2024-04-28T12:38:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.544+0000 7fe755600640 1 --2- 172.21.15.131:0/2660191807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe75015bd60 0x7fe75006f1d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.544+0000 7fe755600640 1 --2- 172.21.15.131:0/2660191807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe75015b980 0x7fe75006ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.544+0000 7fe755600640 1 -- 172.21.15.131:0/2660191807 >> 172.21.15.131:0/2660191807 conn(0x7fe750068090 msgr2=0x7fe7500684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.544+0000 7fe755600640 1 -- 172.21.15.131:0/2660191807 shutdown_connections 2024-04-28T12:38:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.545+0000 7fe755600640 1 -- 172.21.15.131:0/2660191807 wait complete. 2024-04-28T12:38:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.546+0000 7fe755600640 1 Processor -- start 2024-04-28T12:38:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.546+0000 7fe755600640 1 -- start start 2024-04-28T12:38:29.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.547+0000 7fe755600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7500b4610 0x7fe750146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:29.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.547+0000 7fe755600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe75015b980 0x7fe75015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:29.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.547+0000 7fe755600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75015bd60 0x7fe7500ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.547+0000 7fe755600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe75015f6b0 con 0x7fe75015b980 2024-04-28T12:38:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.547+0000 7fe755600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe7500b52b0 con 0x7fe75015bd60 2024-04-28T12:38:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.547+0000 7fe755600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe75015f830 con 0x7fe7500b4610 2024-04-28T12:38:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.547+0000 7fe747e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe75015b980 0x7fe75015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.547+0000 7fe747e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe75015b980 0x7fe75015eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54312/0 (socket says 172.21.15.131:54312) 2024-04-28T12:38:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.547+0000 7fe747e00640 1 -- 172.21.15.131:0/4069891073 learned_addr learned my addr 172.21.15.131:0/4069891073 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.548+0000 7fe747e00640 1 -- 172.21.15.131:0/4069891073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7500b4610 msgr2=0x7fe750146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.548+0000 7fe747e00640 1 --2- 172.21.15.131:0/4069891073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7500b4610 0x7fe750146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.548+0000 7fe747e00640 1 -- 172.21.15.131:0/4069891073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75015bd60 msgr2=0x7fe7500ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.548+0000 7fe747e00640 1 --2- 172.21.15.131:0/4069891073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75015bd60 0x7fe7500ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.548+0000 7fe747e00640 1 -- 172.21.15.131:0/4069891073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7500b2940 con 0x7fe75015b980 2024-04-28T12:38:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.548+0000 7fe747e00640 1 --2- 172.21.15.131:0/4069891073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe75015b980 0x7fe75015eab0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe738035040 tx=0x7fe7380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.548+0000 7fe74cc00640 1 -- 172.21.15.131:0/4069891073 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738037040 con 0x7fe75015b980 2024-04-28T12:38:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.548+0000 7fe74cc00640 1 -- 172.21.15.131:0/4069891073 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe738030770 con 0x7fe75015b980 2024-04-28T12:38:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.548+0000 7fe755600640 1 -- 172.21.15.131:0/4069891073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7500b2c20 con 0x7fe75015b980 2024-04-28T12:38:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.548+0000 7fe74cc00640 1 -- 172.21.15.131:0/4069891073 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738030b80 con 0x7fe75015b980 2024-04-28T12:38:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.549+0000 7fe755600640 1 -- 172.21.15.131:0/4069891073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7500b31e0 con 0x7fe75015b980 2024-04-28T12:38:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.550+0000 7fe755600640 1 -- 172.21.15.131:0/4069891073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7500506c0 con 0x7fe75015b980 2024-04-28T12:38:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.551+0000 7fe74cc00640 1 -- 172.21.15.131:0/4069891073 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe738044020 con 0x7fe75015b980 2024-04-28T12:38:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.552+0000 7fe74cc00640 1 --2- 172.21.15.131:0/4069891073 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe72c06c8f0 0x7fe72c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.552+0000 7fe74cc00640 1 -- 172.21.15.131:0/4069891073 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe7380ac430 con 0x7fe75015b980 2024-04-28T12:38:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.552+0000 7fe74fe00640 1 --2- 172.21.15.131:0/4069891073 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe72c06c8f0 0x7fe72c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.553+0000 7fe74fe00640 1 --2- 172.21.15.131:0/4069891073 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe72c06c8f0 0x7fe72c06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe740006fd0 tx=0x7fe740008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:29.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.556+0000 7fe74cc00640 1 -- 172.21.15.131:0/4069891073 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe738032e20 con 0x7fe75015b980 2024-04-28T12:38:29.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.753+0000 7fe755600640 1 -- 172.21.15.131:0/4069891073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe750159fe0 con 0x7fe75015b980 2024-04-28T12:38:29.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.759+0000 7fe74cc00640 1 -- 172.21.15.131:0/4069891073 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fe73807a4f0 con 0x7fe75015b980 2024-04-28T12:38:29.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.762+0000 7fe755600640 1 -- 172.21.15.131:0/4069891073 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe72c06c8f0 msgr2=0x7fe72c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:29.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.762+0000 7fe755600640 1 --2- 172.21.15.131:0/4069891073 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe72c06c8f0 0x7fe72c06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe740006fd0 tx=0x7fe740008040 comp rx=0 tx=0).stop 2024-04-28T12:38:29.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.762+0000 7fe755600640 1 -- 172.21.15.131:0/4069891073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe75015b980 msgr2=0x7fe75015eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:29.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.762+0000 7fe755600640 1 --2- 172.21.15.131:0/4069891073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe75015b980 0x7fe75015eab0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe738035040 tx=0x7fe7380046c0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.762+0000 7fe755600640 1 -- 172.21.15.131:0/4069891073 shutdown_connections 2024-04-28T12:38:29.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.762+0000 7fe755600640 1 --2- 172.21.15.131:0/4069891073 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe72c06c8f0 0x7fe72c06edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.762+0000 7fe755600640 1 --2- 172.21.15.131:0/4069891073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe75015bd60 0x7fe7500ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.762+0000 7fe755600640 1 --2- 172.21.15.131:0/4069891073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe75015b980 0x7fe75015eab0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.762+0000 7fe755600640 1 --2- 172.21.15.131:0/4069891073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7500b4610 0x7fe750146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:29.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.762+0000 7fe755600640 1 -- 172.21.15.131:0/4069891073 >> 172.21.15.131:0/4069891073 conn(0x7fe750068090 msgr2=0x7fe750156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:29.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.762+0000 7fe755600640 1 -- 172.21.15.131:0/4069891073 shutdown_connections 2024-04-28T12:38:29.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:29.762+0000 7fe755600640 1 -- 172.21.15.131:0/4069891073 wait complete. 2024-04-28T12:38:29.776 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-28T12:38:29.776 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-28T12:38:29.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-28T12:38:29.994 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-28T12:38:29.994 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:38:29.995 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:38:30.049 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:38:30.049 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:38:30.099 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:30.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.342+0000 7f06d1a00640 1 Processor -- start 2024-04-28T12:38:30.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.342+0000 7f06d1a00640 1 -- start start 2024-04-28T12:38:30.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.344+0000 7f06d1a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f06cc1467c0 0x7f06cc072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.344+0000 7f06d1a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f06cc0b3ce0 0x7f06cc0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.344+0000 7f06d1a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06cc0b4140 0x7f06cc0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.344+0000 7f06d1a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f06cc073dd0 con 0x7f06cc0b3ce0 2024-04-28T12:38:30.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.344+0000 7f06d1a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f06cc146ba0 con 0x7f06cc1467c0 2024-04-28T12:38:30.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.344+0000 7f06d1a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f06cc0731c0 con 0x7f06cc0b4140 2024-04-28T12:38:30.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.344+0000 7f06c2000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f06cc0b3ce0 0x7f06cc0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:30.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.344+0000 7f06c2000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f06cc0b3ce0 0x7f06cc0d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54328/0 (socket says 172.21.15.131:54328) 2024-04-28T12:38:30.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.344+0000 7f06c2000640 1 -- 172.21.15.131:0/2659206496 learned_addr learned my addr 172.21.15.131:0/2659206496 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:30.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.345+0000 7f06cb400640 1 --2- 172.21.15.131:0/2659206496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06cc0b4140 0x7f06cc0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:30.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.345+0000 7f06caa00640 1 --2- 172.21.15.131:0/2659206496 >> v2:172.21.15.157:3300/0 conn(0x7f06cc1467c0 0x7f06cc072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:30.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.345+0000 7f06c2000640 1 -- 172.21.15.131:0/2659206496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06cc0b4140 msgr2=0x7f06cc0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:30.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.345+0000 7f06c2000640 1 --2- 172.21.15.131:0/2659206496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06cc0b4140 0x7f06cc0d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.345+0000 7f06c2000640 1 -- 172.21.15.131:0/2659206496 >> v2:172.21.15.157:3300/0 conn(0x7f06cc1467c0 msgr2=0x7f06cc072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:30.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.345+0000 7f06c2000640 1 --2- 172.21.15.131:0/2659206496 >> v2:172.21.15.157:3300/0 conn(0x7f06cc1467c0 0x7f06cc072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.345+0000 7f06c2000640 1 -- 172.21.15.131:0/2659206496 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc0734a0 con 0x7f06cc0b3ce0 2024-04-28T12:38:30.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.345+0000 7f06caa00640 1 --2- 172.21.15.131:0/2659206496 >> v2:172.21.15.157:3300/0 conn(0x7f06cc1467c0 0x7f06cc072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:30.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.346+0000 7f06c2000640 1 --2- 172.21.15.131:0/2659206496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06cc0b3ce0 0x7f06cc0d0b20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f06cc06d150 tx=0x7f06b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a46b955694901e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:30.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.346+0000 7f06ca000640 1 -- 172.21.15.131:0/2659206496 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4037070 con 0x7f06cc0b3ce0 2024-04-28T12:38:30.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.346+0000 7f06ca000640 1 -- 172.21.15.131:0/2659206496 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b4032440 con 0x7f06cc0b3ce0 2024-04-28T12:38:30.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.346+0000 7f06ca000640 1 -- 172.21.15.131:0/2659206496 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4032810 con 0x7f06cc0b3ce0 2024-04-28T12:38:30.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.346+0000 7f06d1a00640 1 -- 172.21.15.131:0/2659206496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06cc0b3ce0 msgr2=0x7f06cc0d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:30.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.346+0000 7f06d1a00640 1 --2- 172.21.15.131:0/2659206496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06cc0b3ce0 0x7f06cc0d0b20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f06cc06d150 tx=0x7f06b402a1e0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.347+0000 7f06d1a00640 1 -- 172.21.15.131:0/2659206496 shutdown_connections 2024-04-28T12:38:30.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.347+0000 7f06d1a00640 1 --2- 172.21.15.131:0/2659206496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06cc0b3ce0 0x7f06cc0d0b20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.347+0000 7f06d1a00640 1 --2- 172.21.15.131:0/2659206496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06cc0b4140 0x7f06cc0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.347+0000 7f06d1a00640 1 --2- 172.21.15.131:0/2659206496 >> v2:172.21.15.157:3300/0 conn(0x7f06cc1467c0 0x7f06cc072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.347+0000 7f06d1a00640 1 -- 172.21.15.131:0/2659206496 >> 172.21.15.131:0/2659206496 conn(0x7f06cc0b2900 msgr2=0x7f06cc0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:30.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.347+0000 7f06d1a00640 1 -- 172.21.15.131:0/2659206496 shutdown_connections 2024-04-28T12:38:30.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.347+0000 7f06d1a00640 1 -- 172.21.15.131:0/2659206496 wait complete. 2024-04-28T12:38:30.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.348+0000 7f06d1a00640 1 Processor -- start 2024-04-28T12:38:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.348+0000 7f06d1a00640 1 -- start start 2024-04-28T12:38:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.349+0000 7f06d1a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06cc0b3ce0 0x7f06cc1f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.349+0000 7f06d1a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06cc1467c0 0x7f06cc1f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.349+0000 7f06d1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06cc1f4ee0 0x7f06cc1e8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.349+0000 7f06d1a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc160a20 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.349+0000 7f06d1a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc073a60 con 0x7f06cc0b3ce0 2024-04-28T12:38:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.349+0000 7f06d1a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f06cc160ba0 con 0x7f06cc1467c0 2024-04-28T12:38:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.350+0000 7f06cb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06cc1f4ee0 0x7f06cc1e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.350+0000 7f06cb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06cc1f4ee0 0x7f06cc1e8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54340/0 (socket says 172.21.15.131:54340) 2024-04-28T12:38:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.350+0000 7f06cb400640 1 -- 172.21.15.131:0/4040590976 learned_addr learned my addr 172.21.15.131:0/4040590976 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.350+0000 7f06caa00640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06cc0b3ce0 0x7f06cc1f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:30.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.350+0000 7f06cb400640 1 -- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06cc1467c0 msgr2=0x7f06cc1f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:30.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.350+0000 7f06cb400640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06cc1467c0 0x7f06cc1f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.350+0000 7f06cb400640 1 -- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06cc0b3ce0 msgr2=0x7f06cc1f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:30.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.350+0000 7f06cb400640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06cc0b3ce0 0x7f06cc1f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.350+0000 7f06cb400640 1 -- 172.21.15.131:0/4040590976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc1f5480 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.350+0000 7f06caa00640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06cc0b3ce0 0x7f06cc1f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:30.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.350+0000 7f06cb400640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06cc1f4ee0 0x7f06cc1e8300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f06c400afd0 tx=0x7f06c400ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:30.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.351+0000 7f06c3400640 1 -- 172.21.15.131:0/4040590976 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4015070 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.351+0000 7f06c3400640 1 -- 172.21.15.131:0/4040590976 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c4004260 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.351+0000 7f06c3400640 1 -- 172.21.15.131:0/4040590976 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4003610 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.351+0000 7f06d1a00640 1 -- 172.21.15.131:0/4040590976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc1f5760 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.351+0000 7f06d1a00640 1 -- 172.21.15.131:0/4040590976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc14b240 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.352+0000 7f06d1a00640 1 -- 172.21.15.131:0/4040590976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06cc069350 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.353+0000 7f06c3400640 1 -- 172.21.15.131:0/4040590976 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06c40037d0 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.354+0000 7f06c3400640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f06a806c8f0 0x7f06a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.354+0000 7f06c3400640 1 -- 172.21.15.131:0/4040590976 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f06c408bd50 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.355+0000 7f06caa00640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f06a806c8f0 0x7f06a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:30.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.356+0000 7f06caa00640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f06a806c8f0 0x7f06a806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f06bc007f40 tx=0x7f06bc007490 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:30.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.359+0000 7f06c3400640 1 -- 172.21.15.131:0/4040590976 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f06c4059ce0 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.560+0000 7f06d1a00640 1 -- 172.21.15.131:0/4040590976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06cc145e20 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.564+0000 7f06c3400640 1 -- 172.21.15.131:0/4040590976 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7f06c40595e0 con 0x7f06cc1f4ee0 2024-04-28T12:38:30.561 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:30.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.566+0000 7f06d1a00640 1 -- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f06a806c8f0 msgr2=0x7f06a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:30.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.566+0000 7f06d1a00640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f06a806c8f0 0x7f06a806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f06bc007f40 tx=0x7f06bc007490 comp rx=0 tx=0).stop 2024-04-28T12:38:30.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.566+0000 7f06d1a00640 1 -- 172.21.15.131:0/4040590976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06cc1f4ee0 msgr2=0x7f06cc1e8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:30.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.566+0000 7f06d1a00640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06cc1f4ee0 0x7f06cc1e8300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f06c400afd0 tx=0x7f06c400ca20 comp rx=0 tx=0).stop 2024-04-28T12:38:30.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.566+0000 7f06d1a00640 1 -- 172.21.15.131:0/4040590976 shutdown_connections 2024-04-28T12:38:30.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.566+0000 7f06d1a00640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f06a806c8f0 0x7f06a806edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.566+0000 7f06d1a00640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06cc1f4ee0 0x7f06cc1e8300 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.566+0000 7f06d1a00640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06cc1467c0 0x7f06cc1f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.566+0000 7f06d1a00640 1 --2- 172.21.15.131:0/4040590976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06cc0b3ce0 0x7f06cc1f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.566+0000 7f06d1a00640 1 -- 172.21.15.131:0/4040590976 >> 172.21.15.131:0/4040590976 conn(0x7f06cc0b2900 msgr2=0x7f06cc144da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:30.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.567+0000 7f06d1a00640 1 -- 172.21.15.131:0/4040590976 shutdown_connections 2024-04-28T12:38:30.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.567+0000 7f06d1a00640 1 -- 172.21.15.131:0/4040590976 wait complete. 2024-04-28T12:38:30.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:30.583 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.791+0000 7f5aeee00640 1 Processor -- start 2024-04-28T12:38:30.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.791+0000 7f5aeee00640 1 -- start start 2024-04-28T12:38:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.792+0000 7f5aeee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ae80721d0 0x7f5ae80b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.792+0000 7f5aeee00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ae8072cc0 0x7f5ae8164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.792+0000 7f5aeee00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ae80737b0 con 0x7f5ae80afea0 2024-04-28T12:38:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.792+0000 7f5aeee00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5ae80b4a70 con 0x7f5ae8072cc0 2024-04-28T12:38:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.793+0000 7f5aeee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae8073a50 con 0x7f5ae80721d0 2024-04-28T12:38:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.793+0000 7f5aee400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ae8072cc0 0x7f5ae8164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.793+0000 7f5aee400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ae8072cc0 0x7f5ae8164de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54346/0 (socket says 172.21.15.131:54346) 2024-04-28T12:38:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.793+0000 7f5aee400640 1 -- 172.21.15.131:0/1956289827 learned_addr learned my addr 172.21.15.131:0/1956289827 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.793+0000 7f5aeda00640 1 --2- 172.21.15.131:0/1956289827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ae80721d0 0x7f5ae80b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.793+0000 7f5aee400640 1 -- 172.21.15.131:0/1956289827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ae80721d0 msgr2=0x7f5ae80b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.793+0000 7f5aee400640 1 --2- 172.21.15.131:0/1956289827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ae80721d0 0x7f5ae80b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.793+0000 7f5aee400640 1 -- 172.21.15.131:0/1956289827 >> v1:172.21.15.157:6789/0 conn(0x7f5ae80afea0 legacy=0x7f5ae80725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.793+0000 7f5aee400640 1 -- 172.21.15.131:0/1956289827 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae8073d30 con 0x7f5ae8072cc0 2024-04-28T12:38:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.795+0000 7f5aee400640 1 --2- 172.21.15.131:0/1956289827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8072cc0 0x7f5ae8164de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009e50 tx=0x7f5ae002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f76666deea5b0c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.795+0000 7f5ae7e00640 1 -- 172.21.15.131:0/1956289827 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0036070 con 0x7f5ae8072cc0 2024-04-28T12:38:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.795+0000 7f5ae7e00640 1 -- 172.21.15.131:0/1956289827 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ae0031600 con 0x7f5ae8072cc0 2024-04-28T12:38:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.795+0000 7f5ae7e00640 1 -- 172.21.15.131:0/1956289827 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae00319d0 con 0x7f5ae8072cc0 2024-04-28T12:38:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.795+0000 7f5aeee00640 1 -- 172.21.15.131:0/1956289827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8072cc0 msgr2=0x7f5ae8164de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.795+0000 7f5aeee00640 1 --2- 172.21.15.131:0/1956289827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8072cc0 0x7f5ae8164de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009e50 tx=0x7f5ae002a710 comp rx=0 tx=0).stop 2024-04-28T12:38:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.796+0000 7f5aeee00640 1 -- 172.21.15.131:0/1956289827 shutdown_connections 2024-04-28T12:38:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.796+0000 7f5aeee00640 1 --2- 172.21.15.131:0/1956289827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8072cc0 0x7f5ae8164de0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.796+0000 7f5aeee00640 1 --2- 172.21.15.131:0/1956289827 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ae80721d0 0x7f5ae80b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.796+0000 7f5aeee00640 1 -- 172.21.15.131:0/1956289827 >> 172.21.15.131:0/1956289827 conn(0x7f5ae8068090 msgr2=0x7f5ae80684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.796+0000 7f5aeee00640 1 -- 172.21.15.131:0/1956289827 shutdown_connections 2024-04-28T12:38:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.796+0000 7f5aeee00640 1 -- 172.21.15.131:0/1956289827 wait complete. 2024-04-28T12:38:30.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.797+0000 7f5aeee00640 1 Processor -- start 2024-04-28T12:38:30.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.797+0000 7f5aeee00640 1 -- start start 2024-04-28T12:38:30.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.798+0000 7f5aeee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae80721d0 0x7f5ae814d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.798+0000 7f5aeee00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8072cc0 0x7f5ae81f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.798+0000 7f5aeee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ae80afea0 0x7f5ae80ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.798+0000 7f5aeee00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae815b630 con 0x7f5ae8072cc0 2024-04-28T12:38:30.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.798+0000 7f5aeee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae80b4550 con 0x7f5ae80721d0 2024-04-28T12:38:30.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.798+0000 7f5aeee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae80cd2b0 con 0x7f5ae80afea0 2024-04-28T12:38:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.799+0000 7f5aed000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8072cc0 0x7f5ae81f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.799+0000 7f5aed000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8072cc0 0x7f5ae81f8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54354/0 (socket says 172.21.15.131:54354) 2024-04-28T12:38:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.799+0000 7f5aed000640 1 -- 172.21.15.131:0/1509940816 learned_addr learned my addr 172.21.15.131:0/1509940816 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.799+0000 7f5aee400640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ae80afea0 0x7f5ae80ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.799+0000 7f5aeda00640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae80721d0 0x7f5ae814d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.799+0000 7f5aed000640 1 -- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ae80afea0 msgr2=0x7f5ae80ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.799+0000 7f5aed000640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ae80afea0 0x7f5ae80ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.799+0000 7f5aed000640 1 -- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae80721d0 msgr2=0x7f5ae814d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.799+0000 7f5aed000640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae80721d0 0x7f5ae814d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.799+0000 7f5aed000640 1 -- 172.21.15.131:0/1509940816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae81f9390 con 0x7f5ae8072cc0 2024-04-28T12:38:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.799+0000 7f5aeda00640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae80721d0 0x7f5ae814d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.799+0000 7f5aed000640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8072cc0 0x7f5ae81f8e50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c340 tx=0x7f5ad400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.800+0000 7f5ae5600640 1 -- 172.21.15.131:0/1509940816 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4017070 con 0x7f5ae8072cc0 2024-04-28T12:38:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.800+0000 7f5ae5600640 1 -- 172.21.15.131:0/1509940816 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad40052d0 con 0x7f5ae8072cc0 2024-04-28T12:38:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.800+0000 7f5ae5600640 1 -- 172.21.15.131:0/1509940816 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4003850 con 0x7f5ae8072cc0 2024-04-28T12:38:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.800+0000 7f5aeee00640 1 -- 172.21.15.131:0/1509940816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae81f5230 con 0x7f5ae8072cc0 2024-04-28T12:38:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.800+0000 7f5aeee00640 1 -- 172.21.15.131:0/1509940816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae81f57f0 con 0x7f5ae8072cc0 2024-04-28T12:38:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.801+0000 7f5aeee00640 1 -- 172.21.15.131:0/1509940816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae80b55c0 con 0x7f5ae8072cc0 2024-04-28T12:38:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.802+0000 7f5ae5600640 1 -- 172.21.15.131:0/1509940816 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ad401a070 con 0x7f5ae8072cc0 2024-04-28T12:38:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.803+0000 7f5ae5600640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ac806c920 0x7f5ac806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.803+0000 7f5ae5600640 1 -- 172.21.15.131:0/1509940816 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5ad408b270 con 0x7f5ae8072cc0 2024-04-28T12:38:30.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.804+0000 7f5aeda00640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ac806c920 0x7f5ac806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:30.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.805+0000 7f5aeda00640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ac806c920 0x7f5ac806ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5adc006fd0 tx=0x7f5adc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:30.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:30.808+0000 7f5ae5600640 1 -- 172.21.15.131:0/1509940816 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ae80b55c0 con 0x7f5ae8072cc0 2024-04-28T12:38:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.012+0000 7f5aeee00640 1 -- 172.21.15.131:0/1509940816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ae80506c0 con 0x7f5ae8072cc0 2024-04-28T12:38:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.013+0000 7f5ae5600640 1 -- 172.21.15.131:0/1509940816 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f5ae81f9390 con 0x7f5ae8072cc0 2024-04-28T12:38:31.010 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:31.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.016+0000 7f5aeee00640 1 -- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ac806c920 msgr2=0x7f5ac806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.016+0000 7f5aeee00640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ac806c920 0x7f5ac806ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5adc006fd0 tx=0x7f5adc008040 comp rx=0 tx=0).stop 2024-04-28T12:38:31.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.016+0000 7f5aeee00640 1 -- 172.21.15.131:0/1509940816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8072cc0 msgr2=0x7f5ae81f8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.016+0000 7f5aeee00640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8072cc0 0x7f5ae81f8e50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c340 tx=0x7f5ad400c800 comp rx=0 tx=0).stop 2024-04-28T12:38:31.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.016+0000 7f5aeee00640 1 -- 172.21.15.131:0/1509940816 shutdown_connections 2024-04-28T12:38:31.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.016+0000 7f5aeee00640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ac806c920 0x7f5ac806ede0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.016+0000 7f5aeee00640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ae80afea0 0x7f5ae80ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.016+0000 7f5aeee00640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8072cc0 0x7f5ae81f8e50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.016+0000 7f5aeee00640 1 --2- 172.21.15.131:0/1509940816 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae80721d0 0x7f5ae814d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.016+0000 7f5aeee00640 1 -- 172.21.15.131:0/1509940816 >> 172.21.15.131:0/1509940816 conn(0x7f5ae8068090 msgr2=0x7f5ae815a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:31.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.016+0000 7f5aeee00640 1 -- 172.21.15.131:0/1509940816 shutdown_connections 2024-04-28T12:38:31.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.016+0000 7f5aeee00640 1 -- 172.21.15.131:0/1509940816 wait complete. 2024-04-28T12:38:31.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:31.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.237+0000 7f4d8ce00640 1 Processor -- start 2024-04-28T12:38:31.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.237+0000 7f4d8ce00640 1 -- start start 2024-04-28T12:38:31.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.238+0000 7f4d8ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d881458e0 0x7f4d881604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.238+0000 7f4d8ce00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4d880b54b0 0x7f4d8815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.239+0000 7f4d8ce00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d880b3d80 con 0x7f4d8806df40 2024-04-28T12:38:31.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.239+0000 7f4d8ce00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4d880b4060 con 0x7f4d880b54b0 2024-04-28T12:38:31.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.239+0000 7f4d8ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d880b4340 con 0x7f4d881458e0 2024-04-28T12:38:31.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.239+0000 7f4d85400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4d8806df40 0x7f4d880b4da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47062/0 (socket says 172.21.15.131:47062) 2024-04-28T12:38:31.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.239+0000 7f4d85400640 1 -- 172.21.15.131:0/3411816783 learned_addr learned my addr 172.21.15.131:0/3411816783 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:31.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.239+0000 7f4d86800640 1 --2- 172.21.15.131:0/3411816783 >> v2:172.21.15.131:3300/0 conn(0x7f4d880b54b0 0x7f4d8815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:31.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.239+0000 7f4d85e00640 1 --2- 172.21.15.131:0/3411816783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d881458e0 0x7f4d881604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:31.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.239+0000 7f4d84a00640 1 -- 172.21.15.131:0/3411816783 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3131092537 0 0) 0x7f4d880b3d80 con 0x7f4d8806df40 2024-04-28T12:38:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.240+0000 7f4d84a00640 1 -- 172.21.15.131:0/3411816783 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d6c003520 con 0x7f4d8806df40 2024-04-28T12:38:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.240+0000 7f4d86800640 1 -- 172.21.15.131:0/3411816783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d881458e0 msgr2=0x7f4d881604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.240+0000 7f4d86800640 1 --2- 172.21.15.131:0/3411816783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d881458e0 0x7f4d881604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.240+0000 7f4d86800640 1 -- 172.21.15.131:0/3411816783 >> v1:172.21.15.131:6789/0 conn(0x7f4d8806df40 legacy=0x7f4d880b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.240+0000 7f4d86800640 1 -- 172.21.15.131:0/3411816783 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d880b4620 con 0x7f4d880b54b0 2024-04-28T12:38:31.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.241+0000 7f4d86800640 1 --2- 172.21.15.131:0/3411816783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d880b54b0 0x7f4d8815d700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009d90 tx=0x7f4d8002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf7a57f6f46c5c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:31.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.241+0000 7f4d84a00640 1 -- 172.21.15.131:0/3411816783 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80031600 con 0x7f4d880b54b0 2024-04-28T12:38:31.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.241+0000 7f4d84a00640 1 -- 172.21.15.131:0/3411816783 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d800317a0 con 0x7f4d880b54b0 2024-04-28T12:38:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.241+0000 7f4d84a00640 1 -- 172.21.15.131:0/3411816783 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80031b70 con 0x7f4d880b54b0 2024-04-28T12:38:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.242+0000 7f4d8ce00640 1 -- 172.21.15.131:0/3411816783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d880b54b0 msgr2=0x7f4d8815d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.242+0000 7f4d8ce00640 1 --2- 172.21.15.131:0/3411816783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d880b54b0 0x7f4d8815d700 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009d90 tx=0x7f4d8002a5d0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.242+0000 7f4d8ce00640 1 -- 172.21.15.131:0/3411816783 shutdown_connections 2024-04-28T12:38:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.242+0000 7f4d8ce00640 1 --2- 172.21.15.131:0/3411816783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d880b54b0 0x7f4d8815d700 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.242+0000 7f4d8ce00640 1 --2- 172.21.15.131:0/3411816783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d881458e0 0x7f4d881604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.242+0000 7f4d8ce00640 1 -- 172.21.15.131:0/3411816783 >> 172.21.15.131:0/3411816783 conn(0x7f4d880b2900 msgr2=0x7f4d880b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.242+0000 7f4d8ce00640 1 -- 172.21.15.131:0/3411816783 shutdown_connections 2024-04-28T12:38:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.242+0000 7f4d8ce00640 1 -- 172.21.15.131:0/3411816783 wait complete. 2024-04-28T12:38:31.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.244+0000 7f4d8ce00640 1 Processor -- start 2024-04-28T12:38:31.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.244+0000 7f4d8ce00640 1 -- start start 2024-04-28T12:38:31.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d8ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d8806df40 0x7f4d8813a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d8ce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d880b54b0 0x7f4d88169830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d8ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d881458e0 0x7f4d88138580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d8ce00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8815ddc0 con 0x7f4d8806df40 2024-04-28T12:38:31.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d8ce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8815dc40 con 0x7f4d881458e0 2024-04-28T12:38:31.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d8ce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8815df40 con 0x7f4d880b54b0 2024-04-28T12:38:31.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d85e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d8806df40 0x7f4d8813a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:31.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d85e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d8806df40 0x7f4d8813a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37400/0 (socket says 172.21.15.131:37400) 2024-04-28T12:38:31.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d85e00640 1 -- 172.21.15.131:0/71046366 learned_addr learned my addr 172.21.15.131:0/71046366 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:31.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d85e00640 1 -- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d880b54b0 msgr2=0x7f4d88169830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d85e00640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d880b54b0 0x7f4d88169830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d85e00640 1 -- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d881458e0 msgr2=0x7f4d88138580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d86800640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d881458e0 0x7f4d88138580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:31.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d85e00640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d881458e0 0x7f4d88138580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.245+0000 7f4d85e00640 1 -- 172.21.15.131:0/71046366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d880ce2a0 con 0x7f4d8806df40 2024-04-28T12:38:31.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.246+0000 7f4d85e00640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d8806df40 0x7f4d8813a0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800d620 tx=0x7f4d7800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:31.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.246+0000 7f4d7e000640 1 -- 172.21.15.131:0/71046366 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800b620 con 0x7f4d8806df40 2024-04-28T12:38:31.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.246+0000 7f4d7e000640 1 -- 172.21.15.131:0/71046366 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d78019070 con 0x7f4d8806df40 2024-04-28T12:38:31.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.246+0000 7f4d7e000640 1 -- 172.21.15.131:0/71046366 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800ba00 con 0x7f4d8806df40 2024-04-28T12:38:31.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.246+0000 7f4d8ce00640 1 -- 172.21.15.131:0/71046366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d88146280 con 0x7f4d8806df40 2024-04-28T12:38:31.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.246+0000 7f4d8ce00640 1 -- 172.21.15.131:0/71046366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d88146840 con 0x7f4d8806df40 2024-04-28T12:38:31.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.247+0000 7f4d8ce00640 1 -- 172.21.15.131:0/71046366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d88069350 con 0x7f4d8806df40 2024-04-28T12:38:31.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.248+0000 7f4d7e000640 1 -- 172.21.15.131:0/71046366 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d7800bba0 con 0x7f4d8806df40 2024-04-28T12:38:31.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.249+0000 7f4d7e000640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d6806c8f0 0x7f4d6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.249+0000 7f4d7e000640 1 -- 172.21.15.131:0/71046366 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4d7808c3e0 con 0x7f4d8806df40 2024-04-28T12:38:31.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.250+0000 7f4d85400640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d6806c8f0 0x7f4d6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:31.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.251+0000 7f4d85400640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d6806c8f0 0x7f4d6806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4d74000f60 tx=0x7f4d740078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:31.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.254+0000 7f4d7e000640 1 -- 172.21.15.131:0/71046366 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d7805a370 con 0x7f4d8806df40 2024-04-28T12:38:31.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.458+0000 7f4d8ce00640 1 -- 172.21.15.131:0/71046366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4d8806fac0 con 0x7f4d8806df40 2024-04-28T12:38:31.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.459+0000 7f4d7e000640 1 -- 172.21.15.131:0/71046366 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f4d78060070 con 0x7f4d8806df40 2024-04-28T12:38:31.456 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.462+0000 7f4d8ce00640 1 -- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d6806c8f0 msgr2=0x7f4d6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.462+0000 7f4d8ce00640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d6806c8f0 0x7f4d6806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4d74000f60 tx=0x7f4d740078f0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.462+0000 7f4d8ce00640 1 -- 172.21.15.131:0/71046366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d8806df40 msgr2=0x7f4d8813a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.462+0000 7f4d8ce00640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d8806df40 0x7f4d8813a0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800d620 tx=0x7f4d7800dae0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.462+0000 7f4d8ce00640 1 -- 172.21.15.131:0/71046366 shutdown_connections 2024-04-28T12:38:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.462+0000 7f4d8ce00640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d6806c8f0 0x7f4d6806edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.462+0000 7f4d8ce00640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d881458e0 0x7f4d88138580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.462+0000 7f4d8ce00640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d880b54b0 0x7f4d88169830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.462+0000 7f4d8ce00640 1 --2- 172.21.15.131:0/71046366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d8806df40 0x7f4d8813a0a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.462+0000 7f4d8ce00640 1 -- 172.21.15.131:0/71046366 >> 172.21.15.131:0/71046366 conn(0x7f4d880b2900 msgr2=0x7f4d880cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:31.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.462+0000 7f4d8ce00640 1 -- 172.21.15.131:0/71046366 shutdown_connections 2024-04-28T12:38:31.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.462+0000 7f4d8ce00640 1 -- 172.21.15.131:0/71046366 wait complete. 2024-04-28T12:38:31.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.682+0000 7fa939200640 1 Processor -- start 2024-04-28T12:38:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.683+0000 7fa939200640 1 -- start start 2024-04-28T12:38:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.684+0000 7fa939200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9340d5a00 0x7fa9340b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.684+0000 7fa939200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9340d08e0 con 0x7fa9340d1f50 2024-04-28T12:38:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.684+0000 7fa939200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9340d0bc0 con 0x7fa93415f1f0 2024-04-28T12:38:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.684+0000 7fa939200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa9340d5e50 con 0x7fa9340d5a00 2024-04-28T12:38:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa931400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa9340d1f50 0x7fa9340b50e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47070/0 (socket says 172.21.15.131:47070) 2024-04-28T12:38:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa931400640 1 -- 172.21.15.131:0/1139017911 learned_addr learned my addr 172.21.15.131:0/1139017911 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa932800640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9340d5a00 0x7fa9340b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1410659593 0 0) 0x7fa9340d08e0 con 0x7fa9340d1f50 2024-04-28T12:38:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa914003520 con 0x7fa9340d1f50 2024-04-28T12:38:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3169860603 0 0) 0x7fa914003520 con 0x7fa9340d1f50 2024-04-28T12:38:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9340d5a00 msgr2=0x7fa9340b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa930a00640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9340d5a00 0x7fa9340b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 >> v1:172.21.15.157:6789/0 conn(0x7fa93415f1f0 legacy=0x7fa9340d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9340d0ea0 con 0x7fa9340d1f50 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fa91c0019f0 con 0x7fa9340d1f50 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 >> v1:172.21.15.131:6789/0 conn(0x7fa9340d1f50 legacy=0x7fa9340b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.685+0000 7fa930a00640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9140039e0 0x7fa914003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa930a00640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa914004800 0x7fa914004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa930a00640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9340d5a00 0x7fa914005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa9340d5e50 con 0x7fa9140039e0 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa91400baf0 con 0x7fa914004800 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa91400bcf0 con 0x7fa9340d5a00 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa931e00640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9140039e0 0x7fa914003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa931400640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa914004800 0x7fa914004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa931e00640 1 -- 172.21.15.131:0/1139017911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9340d5a00 msgr2=0x7fa914005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa932800640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9340d5a00 0x7fa914005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa931e00640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9340d5a00 0x7fa914005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa931e00640 1 -- 172.21.15.131:0/1139017911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa914004800 msgr2=0x7fa914004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:31.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa931e00640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa914004800 0x7fa914004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa931e00640 1 -- 172.21.15.131:0/1139017911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa914003520 con 0x7fa9140039e0 2024-04-28T12:38:31.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.686+0000 7fa931400640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa914004800 0x7fa914004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.687+0000 7fa931e00640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9140039e0 0x7fa914003e00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa924001750 tx=0x7fa914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef71a8be60380a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.688+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924001370 con 0x7fa9140039e0 2024-04-28T12:38:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.688+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9240052d0 con 0x7fa9140039e0 2024-04-28T12:38:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.688+0000 7fa930a00640 1 -- 172.21.15.131:0/1139017911 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92402abe0 con 0x7fa9140039e0 2024-04-28T12:38:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.688+0000 7fa939200640 1 -- 172.21.15.131:0/1139017911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9140039e0 msgr2=0x7fa914003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.688+0000 7fa939200640 1 --2- 172.21.15.131:0/1139017911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9140039e0 0x7fa914003e00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa924001750 tx=0x7fa914005830 comp rx=0 tx=0).stop 2024-04-28T12:38:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.688+0000 7fa932800640 1 -- 172.21.15.131:0/1139017911 reap_dead start 2024-04-28T12:38:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.688+0000 7fa939200640 1 -- 172.21.15.131:0/1139017911 shutdown_connections 2024-04-28T12:38:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.688+0000 7fa939200640 1 -- 172.21.15.131:0/1139017911 >> 172.21.15.131:0/1139017911 conn(0x7fa9340d4ee0 msgr2=0x7fa9340d52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.688+0000 7fa939200640 1 -- 172.21.15.131:0/1139017911 shutdown_connections 2024-04-28T12:38:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.688+0000 7fa939200640 1 -- 172.21.15.131:0/1139017911 wait complete. 2024-04-28T12:38:31.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.690+0000 7fa939200640 1 Processor -- start 2024-04-28T12:38:31.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.690+0000 7fa939200640 1 -- start start 2024-04-28T12:38:31.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.691+0000 7fa939200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9140039e0 0x7fa9340ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.691+0000 7fa939200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9341f2130 0x7fa934160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.691+0000 7fa939200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9341f7fe0 0x7fa9341f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.691+0000 7fa939200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa93415f5d0 con 0x7fa9341f7fe0 2024-04-28T12:38:31.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.691+0000 7fa939200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9340b4bc0 con 0x7fa9140039e0 2024-04-28T12:38:31.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.691+0000 7fa939200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa9340d0500 con 0x7fa9341f2130 2024-04-28T12:38:31.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.691+0000 7fa932800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9341f7fe0 0x7fa9341f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:31.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.692+0000 7fa932800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9341f7fe0 0x7fa9341f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37418/0 (socket says 172.21.15.131:37418) 2024-04-28T12:38:31.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.692+0000 7fa932800640 1 -- 172.21.15.131:0/677338112 learned_addr learned my addr 172.21.15.131:0/677338112 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.692+0000 7fa932800640 1 -- 172.21.15.131:0/677338112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9341f2130 msgr2=0x7fa934160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.692+0000 7fa932800640 1 --2- 172.21.15.131:0/677338112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9341f2130 0x7fa934160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.692+0000 7fa932800640 1 -- 172.21.15.131:0/677338112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9140039e0 msgr2=0x7fa9340ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.692+0000 7fa932800640 1 --2- 172.21.15.131:0/677338112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9140039e0 0x7fa9340ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.692+0000 7fa932800640 1 -- 172.21.15.131:0/677338112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9341615d0 con 0x7fa9341f7fe0 2024-04-28T12:38:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.692+0000 7fa932800640 1 --2- 172.21.15.131:0/677338112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9341f7fe0 0x7fa9341f8460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0093d0 tx=0x7fa92c009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.693+0000 7fa92a000640 1 -- 172.21.15.131:0/677338112 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c00f040 con 0x7fa9341f7fe0 2024-04-28T12:38:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.693+0000 7fa92a000640 1 -- 172.21.15.131:0/677338112 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa92c018070 con 0x7fa9341f7fe0 2024-04-28T12:38:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.693+0000 7fa92a000640 1 -- 172.21.15.131:0/677338112 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c013870 con 0x7fa9341f7fe0 2024-04-28T12:38:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.693+0000 7fa939200640 1 -- 172.21.15.131:0/677338112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa934160d30 con 0x7fa9341f7fe0 2024-04-28T12:38:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.693+0000 7fa939200640 1 -- 172.21.15.131:0/677338112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9341612f0 con 0x7fa9341f7fe0 2024-04-28T12:38:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.694+0000 7fa939200640 1 -- 172.21.15.131:0/677338112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa934069350 con 0x7fa9341f7fe0 2024-04-28T12:38:31.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.695+0000 7fa92a000640 1 -- 172.21.15.131:0/677338112 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa92c013a10 con 0x7fa9341f7fe0 2024-04-28T12:38:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.696+0000 7fa92a000640 1 --2- 172.21.15.131:0/677338112 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa91006c8f0 0x7fa91006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.696+0000 7fa92a000640 1 -- 172.21.15.131:0/677338112 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa92c08b490 con 0x7fa9341f7fe0 2024-04-28T12:38:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.697+0000 7fa931e00640 1 --2- 172.21.15.131:0/677338112 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa91006c8f0 0x7fa91006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:31.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.698+0000 7fa931e00640 1 --2- 172.21.15.131:0/677338112 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa91006c8f0 0x7fa91006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa924032000 tx=0x7fa92402c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:31.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.701+0000 7fa92a000640 1 -- 172.21.15.131:0/677338112 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa92c022020 con 0x7fa9341f7fe0 2024-04-28T12:38:31.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.899+0000 7fa939200640 1 -- 172.21.15.131:0/677338112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa9340b3a90 con 0x7fa9341f7fe0 2024-04-28T12:38:31.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.899+0000 7fa92a000640 1 -- 172.21.15.131:0/677338112 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fa92c059420 con 0x7fa9341f7fe0 2024-04-28T12:38:31.896 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:31.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.902+0000 7fa939200640 1 -- 172.21.15.131:0/677338112 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa91006c8f0 msgr2=0x7fa91006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.902+0000 7fa939200640 1 --2- 172.21.15.131:0/677338112 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa91006c8f0 0x7fa91006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa924032000 tx=0x7fa92402c000 comp rx=0 tx=0).stop 2024-04-28T12:38:31.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.902+0000 7fa939200640 1 -- 172.21.15.131:0/677338112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9341f7fe0 msgr2=0x7fa9341f8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:31.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.902+0000 7fa939200640 1 --2- 172.21.15.131:0/677338112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9341f7fe0 0x7fa9341f8460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0093d0 tx=0x7fa92c009890 comp rx=0 tx=0).stop 2024-04-28T12:38:31.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.902+0000 7fa939200640 1 -- 172.21.15.131:0/677338112 shutdown_connections 2024-04-28T12:38:31.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.902+0000 7fa939200640 1 --2- 172.21.15.131:0/677338112 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa91006c8f0 0x7fa91006edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.902+0000 7fa939200640 1 --2- 172.21.15.131:0/677338112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9341f7fe0 0x7fa9341f8460 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.902+0000 7fa939200640 1 --2- 172.21.15.131:0/677338112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9341f2130 0x7fa934160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.902+0000 7fa939200640 1 --2- 172.21.15.131:0/677338112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9140039e0 0x7fa9340ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:31.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.902+0000 7fa939200640 1 -- 172.21.15.131:0/677338112 >> 172.21.15.131:0/677338112 conn(0x7fa9340d4ee0 msgr2=0x7fa9340d52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:31.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.903+0000 7fa939200640 1 -- 172.21.15.131:0/677338112 shutdown_connections 2024-04-28T12:38:31.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:31.903+0000 7fa939200640 1 -- 172.21.15.131:0/677338112 wait complete. 2024-04-28T12:38:31.918 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:32.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.124+0000 7fd24b200640 1 Processor -- start 2024-04-28T12:38:32.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.124+0000 7fd24b200640 1 -- start start 2024-04-28T12:38:32.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.125+0000 7fd24b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd24c15a7e0 0x7fd24c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.125+0000 7fd24b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd24c15d9c0 0x7fd24c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.126+0000 7fd24b200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd24c0b3900 con 0x7fd24c0b5410 2024-04-28T12:38:32.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.126+0000 7fd24b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd24c15abc0 con 0x7fd24c15a7e0 2024-04-28T12:38:32.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.126+0000 7fd24b200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd24c0b3b70 con 0x7fd24c15d9c0 2024-04-28T12:38:32.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.126+0000 7fd249e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd24c15a7e0 0x7fd24c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:32.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.126+0000 7fd249400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd24c0b5410 0x7fd24c15d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47078/0 (socket says 172.21.15.131:47078) 2024-04-28T12:38:32.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.126+0000 7fd249400640 1 -- 172.21.15.131:0/1186219018 learned_addr learned my addr 172.21.15.131:0/1186219018 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:32.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.126+0000 7fd24a800640 1 --2- 172.21.15.131:0/1186219018 >> v2:172.21.15.157:3300/0 conn(0x7fd24c15d9c0 0x7fd24c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:32.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.126+0000 7fd249e00640 1 -- 172.21.15.131:0/1186219018 >> v2:172.21.15.157:3300/0 conn(0x7fd24c15d9c0 msgr2=0x7fd24c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:32.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.126+0000 7fd249e00640 1 --2- 172.21.15.131:0/1186219018 >> v2:172.21.15.157:3300/0 conn(0x7fd24c15d9c0 0x7fd24c0b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.126+0000 7fd248a00640 1 -- 172.21.15.131:0/1186219018 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1606563961 0 0) 0x7fd24c0b3900 con 0x7fd24c0b5410 2024-04-28T12:38:32.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.126+0000 7fd249e00640 1 -- 172.21.15.131:0/1186219018 >> v1:172.21.15.131:6789/0 conn(0x7fd24c0b5410 legacy=0x7fd24c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:32.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.126+0000 7fd249e00640 1 -- 172.21.15.131:0/1186219018 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c0b3e50 con 0x7fd24c15a7e0 2024-04-28T12:38:32.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.127+0000 7fd249e00640 1 --2- 172.21.15.131:0/1186219018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd24c15a7e0 0x7fd24c0b4650 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0cd1c0 tx=0x7fd23c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ad8c24a6e313f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:32.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.128+0000 7fd248a00640 1 -- 172.21.15.131:0/1186219018 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c037070 con 0x7fd24c15a7e0 2024-04-28T12:38:32.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.128+0000 7fd248a00640 1 -- 172.21.15.131:0/1186219018 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c032440 con 0x7fd24c15a7e0 2024-04-28T12:38:32.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.128+0000 7fd248a00640 1 -- 172.21.15.131:0/1186219018 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c032810 con 0x7fd24c15a7e0 2024-04-28T12:38:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.128+0000 7fd24b200640 1 -- 172.21.15.131:0/1186219018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd24c15a7e0 msgr2=0x7fd24c0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.128+0000 7fd24b200640 1 --2- 172.21.15.131:0/1186219018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd24c15a7e0 0x7fd24c0b4650 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0cd1c0 tx=0x7fd23c02a1a0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.129+0000 7fd24b200640 1 -- 172.21.15.131:0/1186219018 shutdown_connections 2024-04-28T12:38:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.129+0000 7fd24b200640 1 --2- 172.21.15.131:0/1186219018 >> v2:172.21.15.157:3300/0 conn(0x7fd24c15d9c0 0x7fd24c0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.129+0000 7fd24b200640 1 --2- 172.21.15.131:0/1186219018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd24c15a7e0 0x7fd24c0b4650 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.129+0000 7fd24b200640 1 -- 172.21.15.131:0/1186219018 >> 172.21.15.131:0/1186219018 conn(0x7fd24c068090 msgr2=0x7fd24c0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.129+0000 7fd24b200640 1 -- 172.21.15.131:0/1186219018 shutdown_connections 2024-04-28T12:38:32.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.129+0000 7fd24b200640 1 -- 172.21.15.131:0/1186219018 wait complete. 2024-04-28T12:38:32.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.131+0000 7fd24b200640 1 Processor -- start 2024-04-28T12:38:32.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.131+0000 7fd24b200640 1 -- start start 2024-04-28T12:38:32.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.132+0000 7fd24b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd24c0b5410 0x7fd24c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.132+0000 7fd24b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd24c15a7e0 0x7fd24c1e5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.132+0000 7fd24b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd24c15d9c0 0x7fd24c158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.132+0000 7fd24b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c0b4b90 con 0x7fd24c0b5410 2024-04-28T12:38:32.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.132+0000 7fd24b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c15e010 con 0x7fd24c15d9c0 2024-04-28T12:38:32.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.132+0000 7fd24b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd24c15e2e0 con 0x7fd24c15a7e0 2024-04-28T12:38:32.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.132+0000 7fd249e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd24c0b5410 0x7fd24c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:32.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.133+0000 7fd249e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd24c0b5410 0x7fd24c158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37440/0 (socket says 172.21.15.131:37440) 2024-04-28T12:38:32.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.133+0000 7fd249e00640 1 -- 172.21.15.131:0/1135303 learned_addr learned my addr 172.21.15.131:0/1135303 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:32.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.133+0000 7fd249e00640 1 -- 172.21.15.131:0/1135303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd24c15a7e0 msgr2=0x7fd24c1e5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:32.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.133+0000 7fd249e00640 1 --2- 172.21.15.131:0/1135303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd24c15a7e0 0x7fd24c1e5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.133+0000 7fd249e00640 1 -- 172.21.15.131:0/1135303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd24c15d9c0 msgr2=0x7fd24c158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:32.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.133+0000 7fd249e00640 1 --2- 172.21.15.131:0/1135303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd24c15d9c0 0x7fd24c158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.133+0000 7fd249e00640 1 -- 172.21.15.131:0/1135303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c1e6270 con 0x7fd24c0b5410 2024-04-28T12:38:32.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.133+0000 7fd249e00640 1 --2- 172.21.15.131:0/1135303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd24c0b5410 0x7fd24c158f80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009890 tx=0x7fd23c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:32.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.134+0000 7fd242000640 1 -- 172.21.15.131:0/1135303 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c037040 con 0x7fd24c0b5410 2024-04-28T12:38:32.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.134+0000 7fd242000640 1 -- 172.21.15.131:0/1135303 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c042070 con 0x7fd24c0b5410 2024-04-28T12:38:32.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.134+0000 7fd242000640 1 -- 172.21.15.131:0/1135303 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c030bd0 con 0x7fd24c0b5410 2024-04-28T12:38:32.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.134+0000 7fd24b200640 1 -- 172.21.15.131:0/1135303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24c1e6550 con 0x7fd24c0b5410 2024-04-28T12:38:32.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.134+0000 7fd24b200640 1 -- 172.21.15.131:0/1135303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24c0ad480 con 0x7fd24c0b5410 2024-04-28T12:38:32.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.135+0000 7fd24b200640 1 -- 172.21.15.131:0/1135303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24c0506c0 con 0x7fd24c0b5410 2024-04-28T12:38:32.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.136+0000 7fd242000640 1 -- 172.21.15.131:0/1135303 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd23c047460 con 0x7fd24c0b5410 2024-04-28T12:38:32.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.137+0000 7fd242000640 1 --2- 172.21.15.131:0/1135303 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd22c06c8f0 0x7fd22c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.137+0000 7fd242000640 1 -- 172.21.15.131:0/1135303 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd23c0acbd0 con 0x7fd24c0b5410 2024-04-28T12:38:32.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.137+0000 7fd249400640 1 --2- 172.21.15.131:0/1135303 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd22c06c8f0 0x7fd22c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:32.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.139+0000 7fd249400640 1 --2- 172.21.15.131:0/1135303 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd22c06c8f0 0x7fd22c06edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd238007fd0 tx=0x7fd238007920 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:32.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.142+0000 7fd242000640 1 -- 172.21.15.131:0/1135303 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd23c032e20 con 0x7fd24c0b5410 2024-04-28T12:38:32.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.343+0000 7fd24b200640 1 -- 172.21.15.131:0/1135303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd24c0ad270 con 0x7fd24c0b5410 2024-04-28T12:38:32.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.344+0000 7fd242000640 1 -- 172.21.15.131:0/1135303 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd23c07ab60 con 0x7fd24c0b5410 2024-04-28T12:38:32.341 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:32.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.347+0000 7fd24b200640 1 -- 172.21.15.131:0/1135303 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd22c06c8f0 msgr2=0x7fd22c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:32.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.347+0000 7fd24b200640 1 --2- 172.21.15.131:0/1135303 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd22c06c8f0 0x7fd22c06edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd238007fd0 tx=0x7fd238007920 comp rx=0 tx=0).stop 2024-04-28T12:38:32.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.347+0000 7fd24b200640 1 -- 172.21.15.131:0/1135303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd24c0b5410 msgr2=0x7fd24c158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:32.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.347+0000 7fd24b200640 1 --2- 172.21.15.131:0/1135303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd24c0b5410 0x7fd24c158f80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009890 tx=0x7fd23c0046c0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.347+0000 7fd24b200640 1 -- 172.21.15.131:0/1135303 shutdown_connections 2024-04-28T12:38:32.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.347+0000 7fd24b200640 1 --2- 172.21.15.131:0/1135303 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd22c06c8f0 0x7fd22c06edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.347+0000 7fd24b200640 1 --2- 172.21.15.131:0/1135303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd24c15d9c0 0x7fd24c158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.347+0000 7fd24b200640 1 --2- 172.21.15.131:0/1135303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd24c15a7e0 0x7fd24c1e5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.347+0000 7fd24b200640 1 --2- 172.21.15.131:0/1135303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd24c0b5410 0x7fd24c158f80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.347+0000 7fd24b200640 1 -- 172.21.15.131:0/1135303 >> 172.21.15.131:0/1135303 conn(0x7fd24c068090 msgr2=0x7fd24c0ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:32.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.347+0000 7fd24b200640 1 -- 172.21.15.131:0/1135303 shutdown_connections 2024-04-28T12:38:32.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.348+0000 7fd24b200640 1 -- 172.21.15.131:0/1135303 wait complete. 2024-04-28T12:38:32.362 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T12:38:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.560+0000 7f0b7f200640 1 Processor -- start 2024-04-28T12:38:32.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.561+0000 7f0b7f200640 1 -- start start 2024-04-28T12:38:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.562+0000 7f0b7f200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0b800d1620 0x7f0b800d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.562+0000 7f0b7f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b800d1a00 0x7f0b800b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.562+0000 7f0b7f200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b800d0480 con 0x7f0b800b50b0 2024-04-28T12:38:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.562+0000 7f0b7f200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0b800d0ce0 con 0x7f0b800d1620 2024-04-28T12:38:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.562+0000 7f0b7f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0b800d0720 con 0x7f0b800d1a00 2024-04-28T12:38:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.562+0000 7f0b7e800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0b800b50b0 0x7f0b800b5540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47084/0 (socket says 172.21.15.131:47084) 2024-04-28T12:38:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.562+0000 7f0b7e800640 1 -- 172.21.15.131:0/4086385832 learned_addr learned my addr 172.21.15.131:0/4086385832 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.562+0000 7f0b7de00640 1 --2- 172.21.15.131:0/4086385832 >> v2:172.21.15.157:3300/0 conn(0x7f0b800d1620 0x7f0b800d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1906334976 0 0) 0x7f0b800d0480 con 0x7f0b800b50b0 2024-04-28T12:38:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b60003520 con 0x7f0b800b50b0 2024-04-28T12:38:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7d400640 1 --2- 172.21.15.131:0/4086385832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b800d1a00 0x7f0b800b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3070483137 0 0) 0x7f0b60003520 con 0x7f0b800b50b0 2024-04-28T12:38:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b800d1a00 msgr2=0x7f0b800b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 --2- 172.21.15.131:0/4086385832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b800d1a00 0x7f0b800b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 >> v2:172.21.15.157:3300/0 conn(0x7f0b800d1620 msgr2=0x7f0b800d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 --2- 172.21.15.131:0/4086385832 >> v2:172.21.15.157:3300/0 conn(0x7f0b800d1620 0x7f0b800d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b800d0a00 con 0x7f0b800b50b0 2024-04-28T12:38:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f0b78001670 con 0x7f0b800b50b0 2024-04-28T12:38:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 >> v1:172.21.15.131:6789/0 conn(0x7f0b800b50b0 legacy=0x7f0b800b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 --2- 172.21.15.131:0/4086385832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b600039b0 0x7f0b60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 --2- 172.21.15.131:0/4086385832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b800d1a00 0x7f0b60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 --2- 172.21.15.131:0/4086385832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b60004d40 0x7f0b600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0b800d0ce0 con 0x7f0b600039b0 2024-04-28T12:38:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0b800d0720 con 0x7f0b60004d40 2024-04-28T12:38:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6000baa0 con 0x7f0b800d1a00 2024-04-28T12:38:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.563+0000 7f0b7de00640 1 --2- 172.21.15.131:0/4086385832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b600039b0 0x7f0b60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.564+0000 7f0b7de00640 1 -- 172.21.15.131:0/4086385832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b800d1a00 msgr2=0x7f0b60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.564+0000 7f0b7de00640 1 --2- 172.21.15.131:0/4086385832 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b800d1a00 0x7f0b60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.564+0000 7f0b7de00640 1 -- 172.21.15.131:0/4086385832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b60004d40 msgr2=0x7f0b600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.564+0000 7f0b7de00640 1 --2- 172.21.15.131:0/4086385832 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b60004d40 0x7f0b600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.564+0000 7f0b7de00640 1 -- 172.21.15.131:0/4086385832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b60003520 con 0x7f0b600039b0 2024-04-28T12:38:32.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.565+0000 7f0b7de00640 1 --2- 172.21.15.131:0/4086385832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b600039b0 0x7f0b60003dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009ac0 tx=0x7f0b60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5207bda7ac86a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.565+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70036070 con 0x7f0b600039b0 2024-04-28T12:38:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.565+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b70031400 con 0x7f0b600039b0 2024-04-28T12:38:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.565+0000 7f0b7ca00640 1 -- 172.21.15.131:0/4086385832 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70009690 con 0x7f0b600039b0 2024-04-28T12:38:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.566+0000 7f0b7f200640 1 -- 172.21.15.131:0/4086385832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b600039b0 msgr2=0x7f0b60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.566+0000 7f0b7f200640 1 --2- 172.21.15.131:0/4086385832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b600039b0 0x7f0b60003dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009ac0 tx=0x7f0b60005830 comp rx=0 tx=0).stop 2024-04-28T12:38:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.566+0000 7f0b7e800640 1 -- 172.21.15.131:0/4086385832 reap_dead start 2024-04-28T12:38:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.566+0000 7f0b7f200640 1 -- 172.21.15.131:0/4086385832 shutdown_connections 2024-04-28T12:38:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.566+0000 7f0b7f200640 1 -- 172.21.15.131:0/4086385832 >> 172.21.15.131:0/4086385832 conn(0x7f0b800d4c30 msgr2=0x7f0b800d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.566+0000 7f0b7f200640 1 -- 172.21.15.131:0/4086385832 shutdown_connections 2024-04-28T12:38:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.566+0000 7f0b7f200640 1 -- 172.21.15.131:0/4086385832 wait complete. 2024-04-28T12:38:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.568+0000 7f0b7f200640 1 Processor -- start 2024-04-28T12:38:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.568+0000 7f0b7f200640 1 -- start start 2024-04-28T12:38:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b600039b0 0x7f0b800c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b800cb1d0 0x7f0b800b7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80150010 0x7f0b800b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0b800b5c50 con 0x7f0b80150010 2024-04-28T12:38:32.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0b800d1e60 con 0x7f0b600039b0 2024-04-28T12:38:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0b800b5dd0 con 0x7f0b800cb1d0 2024-04-28T12:38:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80150010 0x7f0b800b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80150010 0x7f0b800b6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37460/0 (socket says 172.21.15.131:37460) 2024-04-28T12:38:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7e800640 1 -- 172.21.15.131:0/4106722468 learned_addr learned my addr 172.21.15.131:0/4106722468 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7de00640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b600039b0 0x7f0b800c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7d400640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b800cb1d0 0x7f0b800b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7e800640 1 -- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b800cb1d0 msgr2=0x7f0b800b7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:32.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7e800640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b800cb1d0 0x7f0b800b7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7e800640 1 -- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b600039b0 msgr2=0x7f0b800c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7e800640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b600039b0 0x7f0b800c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.569+0000 7f0b7e800640 1 -- 172.21.15.131:0/4106722468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b800b7e80 con 0x7f0b80150010 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.570+0000 7f0b7e800640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80150010 0x7f0b800b6040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0b800d2730 tx=0x7f0b7800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.570+0000 7f0b76000640 1 -- 172.21.15.131:0/4106722468 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78007440 con 0x7f0b80150010 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.570+0000 7f0b76000640 1 -- 172.21.15.131:0/4106722468 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b780075e0 con 0x7f0b80150010 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.570+0000 7f0b76000640 1 -- 172.21.15.131:0/4106722468 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78004a20 con 0x7f0b80150010 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.570+0000 7f0b7f200640 1 -- 172.21.15.131:0/4106722468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b800ca130 con 0x7f0b80150010 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.570+0000 7f0b7f200640 1 -- 172.21.15.131:0/4106722468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b800ca6f0 con 0x7f0b80150010 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.571+0000 7f0b7f200640 1 -- 172.21.15.131:0/4106722468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b80069350 con 0x7f0b80150010 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.572+0000 7f0b76000640 1 -- 172.21.15.131:0/4106722468 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b78011070 con 0x7f0b80150010 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.573+0000 7f0b76000640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:32.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.573+0000 7f0b76000640 1 -- 172.21.15.131:0/4106722468 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0b7808b640 con 0x7f0b80150010 2024-04-28T12:38:32.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.574+0000 7f0b7de00640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:32.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.575+0000 7f0b7de00640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0b70033000 tx=0x7f0b70008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:32.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.578+0000 7f0b76000640 1 -- 172.21.15.131:0/4106722468 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b780595d0 con 0x7f0b80150010 2024-04-28T12:38:32.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.791+0000 7f0b7f200640 1 -- 172.21.15.131:0/4106722468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0b800b29f0 con 0x7f0b80150010 2024-04-28T12:38:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.792+0000 7f0b76000640 1 -- 172.21.15.131:0/4106722468 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f0b78058ed0 con 0x7f0b80150010 2024-04-28T12:38:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-04-28T12:38:32.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.794+0000 7f0b7f200640 1 -- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0b5c06c8f0 msgr2=0x7f0b5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:32.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.794+0000 7f0b7f200640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0b70033000 tx=0x7f0b70008030 comp rx=0 tx=0).stop 2024-04-28T12:38:32.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.794+0000 7f0b7f200640 1 -- 172.21.15.131:0/4106722468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80150010 msgr2=0x7f0b800b6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:32.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.794+0000 7f0b7f200640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80150010 0x7f0b800b6040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0b800d2730 tx=0x7f0b7800c960 comp rx=0 tx=0).stop 2024-04-28T12:38:32.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.794+0000 7f0b7f200640 1 -- 172.21.15.131:0/4106722468 shutdown_connections 2024-04-28T12:38:32.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.794+0000 7f0b7f200640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.794+0000 7f0b7f200640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b80150010 0x7f0b800b6040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.794+0000 7f0b7f200640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b800cb1d0 0x7f0b800b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.794+0000 7f0b7f200640 1 --2- 172.21.15.131:0/4106722468 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b600039b0 0x7f0b800c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:32.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.794+0000 7f0b7f200640 1 -- 172.21.15.131:0/4106722468 >> 172.21.15.131:0/4106722468 conn(0x7f0b800d4c30 msgr2=0x7f0b800d3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:32.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.795+0000 7f0b7f200640 1 -- 172.21.15.131:0/4106722468 shutdown_connections 2024-04-28T12:38:32.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:32.795+0000 7f0b7f200640 1 -- 172.21.15.131:0/4106722468 wait complete. 2024-04-28T12:38:32.811 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:38:32.811 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:38:32.861 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:33.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.111+0000 7f839ac00640 1 Processor -- start 2024-04-28T12:38:33.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.111+0000 7f839ac00640 1 -- start start 2024-04-28T12:38:33.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.113+0000 7f839ac00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f839415cab0 0x7f83940afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:33.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.113+0000 7f839ac00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f83940b0020 0x7f83940d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:33.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.113+0000 7f839ac00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83940b0650 con 0x7f83940d1080 2024-04-28T12:38:33.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.113+0000 7f839ac00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f83940d0900 con 0x7f839415cab0 2024-04-28T12:38:33.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.113+0000 7f839ac00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f83941635a0 con 0x7f83940b0020 2024-04-28T12:38:33.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.113+0000 7f8393e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f839415cab0 0x7f83940afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:33.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.114+0000 7f8393e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f839415cab0 0x7f83940afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37462/0 (socket says 172.21.15.131:37462) 2024-04-28T12:38:33.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.114+0000 7f8393e00640 1 -- 172.21.15.131:0/3161311237 learned_addr learned my addr 172.21.15.131:0/3161311237 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:33.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.114+0000 7f8393e00640 1 -- 172.21.15.131:0/3161311237 >> v2:172.21.15.157:3300/0 conn(0x7f83940b0020 msgr2=0x7f83940d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:33.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.114+0000 7f8393e00640 1 --2- 172.21.15.131:0/3161311237 >> v2:172.21.15.157:3300/0 conn(0x7f83940b0020 0x7f83940d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.114+0000 7f8393e00640 1 -- 172.21.15.131:0/3161311237 >> v1:172.21.15.157:6789/0 conn(0x7f83940d1080 legacy=0x7f83940ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:33.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.114+0000 7f8393e00640 1 -- 172.21.15.131:0/3161311237 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8394163a30 con 0x7f839415cab0 2024-04-28T12:38:33.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.115+0000 7f8393e00640 1 --2- 172.21.15.131:0/3161311237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f839415cab0 0x7f83940afae0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f839406ff40 tx=0x7f838402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce4341f40a39f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:33.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.115+0000 7f8393400640 1 -- 172.21.15.131:0/3161311237 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384037070 con 0x7f839415cab0 2024-04-28T12:38:33.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.115+0000 7f8393400640 1 -- 172.21.15.131:0/3161311237 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8384032440 con 0x7f839415cab0 2024-04-28T12:38:33.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.115+0000 7f8393400640 1 -- 172.21.15.131:0/3161311237 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384032810 con 0x7f839415cab0 2024-04-28T12:38:33.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.116+0000 7f839ac00640 1 -- 172.21.15.131:0/3161311237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f839415cab0 msgr2=0x7f83940afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:33.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.116+0000 7f839ac00640 1 --2- 172.21.15.131:0/3161311237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f839415cab0 0x7f83940afae0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f839406ff40 tx=0x7f838402a230 comp rx=0 tx=0).stop 2024-04-28T12:38:33.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.116+0000 7f839ac00640 1 -- 172.21.15.131:0/3161311237 shutdown_connections 2024-04-28T12:38:33.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.116+0000 7f839ac00640 1 --2- 172.21.15.131:0/3161311237 >> v2:172.21.15.157:3300/0 conn(0x7f83940b0020 0x7f83940d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.116+0000 7f839ac00640 1 --2- 172.21.15.131:0/3161311237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f839415cab0 0x7f83940afae0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.116+0000 7f839ac00640 1 -- 172.21.15.131:0/3161311237 >> 172.21.15.131:0/3161311237 conn(0x7f83940b2900 msgr2=0x7f83940b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:33.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.116+0000 7f839ac00640 1 -- 172.21.15.131:0/3161311237 shutdown_connections 2024-04-28T12:38:33.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.116+0000 7f839ac00640 1 -- 172.21.15.131:0/3161311237 wait complete. 2024-04-28T12:38:33.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.118+0000 7f839ac00640 1 Processor -- start 2024-04-28T12:38:33.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.118+0000 7f839ac00640 1 -- start start 2024-04-28T12:38:33.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.119+0000 7f839ac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83940b0020 0x7f839414a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:33.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.119+0000 7f839ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83940d1080 0x7f839415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:33.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.119+0000 7f839ac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f839415cab0 0x7f839415e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:33.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.119+0000 7f839ac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8394162f50 con 0x7f839415cab0 2024-04-28T12:38:33.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.119+0000 7f839ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83940d14b0 con 0x7f83940d1080 2024-04-28T12:38:33.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.119+0000 7f839ac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8394163230 con 0x7f83940b0020 2024-04-28T12:38:33.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.119+0000 7f8398c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f839415cab0 0x7f839415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:33.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.120+0000 7f8398c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f839415cab0 0x7f839415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37466/0 (socket says 172.21.15.131:37466) 2024-04-28T12:38:33.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.120+0000 7f8398c00640 1 -- 172.21.15.131:0/3275660074 learned_addr learned my addr 172.21.15.131:0/3275660074 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:33.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.120+0000 7f838b400640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83940d1080 0x7f839415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:33.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.120+0000 7f8398c00640 1 -- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83940b0020 msgr2=0x7f839414a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:33.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.120+0000 7f8398c00640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83940b0020 0x7f839414a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.120+0000 7f8398c00640 1 -- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83940d1080 msgr2=0x7f839415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:33.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.120+0000 7f8398c00640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83940d1080 0x7f839415e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.120+0000 7f8398c00640 1 -- 172.21.15.131:0/3275660074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83941f5050 con 0x7f839415cab0 2024-04-28T12:38:33.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.120+0000 7f8398c00640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f839415cab0 0x7f839415e720 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c7b0 tx=0x7f838c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:33.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.121+0000 7f8390c00640 1 -- 172.21.15.131:0/3275660074 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c017070 con 0x7f839415cab0 2024-04-28T12:38:33.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.121+0000 7f8390c00640 1 -- 172.21.15.131:0/3275660074 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838c003da0 con 0x7f839415cab0 2024-04-28T12:38:33.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.121+0000 7f8390c00640 1 -- 172.21.15.131:0/3275660074 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c0052b0 con 0x7f839415cab0 2024-04-28T12:38:33.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.121+0000 7f839ac00640 1 -- 172.21.15.131:0/3275660074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83941f52d0 con 0x7f839415cab0 2024-04-28T12:38:33.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.121+0000 7f839ac00640 1 -- 172.21.15.131:0/3275660074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83941f5890 con 0x7f839415cab0 2024-04-28T12:38:33.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.122+0000 7f839ac00640 1 -- 172.21.15.131:0/3275660074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8394069350 con 0x7f839415cab0 2024-04-28T12:38:33.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.123+0000 7f8390c00640 1 -- 172.21.15.131:0/3275660074 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f838c005450 con 0x7f839415cab0 2024-04-28T12:38:33.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.124+0000 7f8390c00640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f837006c8f0 0x7f837006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:33.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.124+0000 7f8390c00640 1 -- 172.21.15.131:0/3275660074 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f838c08b530 con 0x7f839415cab0 2024-04-28T12:38:33.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.124+0000 7f8393e00640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f837006c8f0 0x7f837006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:33.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.125+0000 7f8393e00640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f837006c8f0 0x7f837006edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8384034000 tx=0x7f838402c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:33.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.128+0000 7f8390c00640 1 -- 172.21.15.131:0/3275660074 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f83941f5050 con 0x7f839415cab0 2024-04-28T12:38:33.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.331+0000 7f839ac00640 1 -- 172.21.15.131:0/3275660074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83940afa00 con 0x7f839415cab0 2024-04-28T12:38:33.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.338+0000 7f8390c00640 1 -- 172.21.15.131:0/3275660074 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7f838c059550 con 0x7f839415cab0 2024-04-28T12:38:33.336 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:33.336 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:33.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.341+0000 7f839ac00640 1 -- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f837006c8f0 msgr2=0x7f837006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:33.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.341+0000 7f839ac00640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f837006c8f0 0x7f837006edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8384034000 tx=0x7f838402c000 comp rx=0 tx=0).stop 2024-04-28T12:38:33.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.341+0000 7f839ac00640 1 -- 172.21.15.131:0/3275660074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f839415cab0 msgr2=0x7f839415e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:33.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.341+0000 7f839ac00640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f839415cab0 0x7f839415e720 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c7b0 tx=0x7f838c00cc70 comp rx=0 tx=0).stop 2024-04-28T12:38:33.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.341+0000 7f839ac00640 1 -- 172.21.15.131:0/3275660074 shutdown_connections 2024-04-28T12:38:33.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.341+0000 7f839ac00640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f837006c8f0 0x7f837006edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.341+0000 7f839ac00640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f839415cab0 0x7f839415e720 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.341+0000 7f839ac00640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83940d1080 0x7f839415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.341+0000 7f839ac00640 1 --2- 172.21.15.131:0/3275660074 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83940b0020 0x7f839414a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.341+0000 7f839ac00640 1 -- 172.21.15.131:0/3275660074 >> 172.21.15.131:0/3275660074 conn(0x7f83940b2900 msgr2=0x7f83940b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:33.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.342+0000 7f839ac00640 1 -- 172.21.15.131:0/3275660074 shutdown_connections 2024-04-28T12:38:33.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.342+0000 7f839ac00640 1 -- 172.21.15.131:0/3275660074 wait complete. 2024-04-28T12:38:33.359 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:33.360 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:33.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.565+0000 7efc2f400640 1 Processor -- start 2024-04-28T12:38:33.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.566+0000 7efc2f400640 1 -- start start 2024-04-28T12:38:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.567+0000 7efc2f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2815f350 0x7efc280d3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.567+0000 7efc2f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efc2815f730 0x7efc280d3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.567+0000 7efc2f400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc280b8cd0 con 0x7efc280b87c0 2024-04-28T12:38:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.567+0000 7efc2f400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7efc280b8fb0 con 0x7efc2815f730 2024-04-28T12:38:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.567+0000 7efc2f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc280b9290 con 0x7efc2815f350 2024-04-28T12:38:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.568+0000 7efc1fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7efc280b87c0 0x7efc280cfb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47096/0 (socket says 172.21.15.131:47096) 2024-04-28T12:38:33.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.568+0000 7efc1fe00640 1 -- 172.21.15.131:0/728058507 learned_addr learned my addr 172.21.15.131:0/728058507 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:33.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.568+0000 7efc2d400640 1 --2- 172.21.15.131:0/728058507 >> v2:172.21.15.131:3300/0 conn(0x7efc2815f730 0x7efc280d3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:33.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.568+0000 7efc2ca00640 1 --2- 172.21.15.131:0/728058507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2815f350 0x7efc280d3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:33.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.568+0000 7efc27e00640 1 -- 172.21.15.131:0/728058507 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 222861525 0 0) 0x7efc280b8cd0 con 0x7efc280b87c0 2024-04-28T12:38:33.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.568+0000 7efc27e00640 1 -- 172.21.15.131:0/728058507 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc0c003520 con 0x7efc280b87c0 2024-04-28T12:38:33.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.568+0000 7efc2d400640 1 -- 172.21.15.131:0/728058507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2815f350 msgr2=0x7efc280d3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:33.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.568+0000 7efc2d400640 1 --2- 172.21.15.131:0/728058507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2815f350 0x7efc280d3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.568+0000 7efc2d400640 1 -- 172.21.15.131:0/728058507 >> v1:172.21.15.131:6789/0 conn(0x7efc280b87c0 legacy=0x7efc280cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:33.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.568+0000 7efc2d400640 1 -- 172.21.15.131:0/728058507 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc280b9570 con 0x7efc2815f730 2024-04-28T12:38:33.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.569+0000 7efc2d400640 1 --2- 172.21.15.131:0/728058507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc2815f730 0x7efc280d3a40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efc20009d90 tx=0x7efc2002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112352570758a561 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:33.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.570+0000 7efc27e00640 1 -- 172.21.15.131:0/728058507 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20007740 con 0x7efc2815f730 2024-04-28T12:38:33.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.570+0000 7efc27e00640 1 -- 172.21.15.131:0/728058507 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc200078e0 con 0x7efc2815f730 2024-04-28T12:38:33.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.570+0000 7efc27e00640 1 -- 172.21.15.131:0/728058507 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20007cb0 con 0x7efc2815f730 2024-04-28T12:38:33.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.570+0000 7efc2f400640 1 -- 172.21.15.131:0/728058507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc2815f730 msgr2=0x7efc280d3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:33.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.570+0000 7efc2f400640 1 --2- 172.21.15.131:0/728058507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc2815f730 0x7efc280d3a40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efc20009d90 tx=0x7efc2002a660 comp rx=0 tx=0).stop 2024-04-28T12:38:33.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.570+0000 7efc2f400640 1 -- 172.21.15.131:0/728058507 shutdown_connections 2024-04-28T12:38:33.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.570+0000 7efc2f400640 1 --2- 172.21.15.131:0/728058507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc2815f730 0x7efc280d3a40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.570+0000 7efc2f400640 1 --2- 172.21.15.131:0/728058507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2815f350 0x7efc280d3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.570+0000 7efc2f400640 1 -- 172.21.15.131:0/728058507 >> 172.21.15.131:0/728058507 conn(0x7efc280d4fb0 msgr2=0x7efc280d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:33.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.571+0000 7efc2f400640 1 -- 172.21.15.131:0/728058507 shutdown_connections 2024-04-28T12:38:33.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.571+0000 7efc2f400640 1 -- 172.21.15.131:0/728058507 wait complete. 2024-04-28T12:38:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.572+0000 7efc2f400640 1 Processor -- start 2024-04-28T12:38:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.572+0000 7efc2f400640 1 -- start start 2024-04-28T12:38:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.573+0000 7efc2f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc280b87c0 0x7efc280cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.573+0000 7efc2f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc2815f350 0x7efc2809ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.573+0000 7efc2f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2815f730 0x7efc281f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.573+0000 7efc2f400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efc280d0d20 con 0x7efc280b87c0 2024-04-28T12:38:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.573+0000 7efc2f400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc280bad80 con 0x7efc2815f350 2024-04-28T12:38:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.573+0000 7efc2f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc280d0ea0 con 0x7efc2815f730 2024-04-28T12:38:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.574+0000 7efc2ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc280b87c0 0x7efc280cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.574+0000 7efc2ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc280b87c0 0x7efc280cb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37478/0 (socket says 172.21.15.131:37478) 2024-04-28T12:38:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.574+0000 7efc2ca00640 1 -- 172.21.15.131:0/531199860 learned_addr learned my addr 172.21.15.131:0/531199860 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.574+0000 7efc1fe00640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc2815f350 0x7efc2809ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.574+0000 7efc2ca00640 1 -- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2815f730 msgr2=0x7efc281f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.574+0000 7efc2ca00640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2815f730 0x7efc281f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.574+0000 7efc2ca00640 1 -- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc2815f350 msgr2=0x7efc2809ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.574+0000 7efc2ca00640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc2815f350 0x7efc2809ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.574+0000 7efc2ca00640 1 -- 172.21.15.131:0/531199860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc281f8f10 con 0x7efc280b87c0 2024-04-28T12:38:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.574+0000 7efc2ca00640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc280b87c0 0x7efc280cb140 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efc1800d5f0 tx=0x7efc1800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.575+0000 7efc25600640 1 -- 172.21.15.131:0/531199860 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc180125f0 con 0x7efc280b87c0 2024-04-28T12:38:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.575+0000 7efc25600640 1 -- 172.21.15.131:0/531199860 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc18019070 con 0x7efc280b87c0 2024-04-28T12:38:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.575+0000 7efc2f400640 1 -- 172.21.15.131:0/531199860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc281f8150 con 0x7efc280b87c0 2024-04-28T12:38:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.575+0000 7efc25600640 1 -- 172.21.15.131:0/531199860 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc180129d0 con 0x7efc280b87c0 2024-04-28T12:38:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.575+0000 7efc2f400640 1 -- 172.21.15.131:0/531199860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc281f8710 con 0x7efc280b87c0 2024-04-28T12:38:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.576+0000 7efc2f400640 1 -- 172.21.15.131:0/531199860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc28069350 con 0x7efc280b87c0 2024-04-28T12:38:33.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.577+0000 7efc25600640 1 -- 172.21.15.131:0/531199860 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc18012b70 con 0x7efc280b87c0 2024-04-28T12:38:33.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.578+0000 7efc25600640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efc0406c8f0 0x7efc0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:33.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.578+0000 7efc25600640 1 -- 172.21.15.131:0/531199860 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7efc1808c400 con 0x7efc280b87c0 2024-04-28T12:38:33.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.578+0000 7efc1fe00640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efc0406c8f0 0x7efc0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:33.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.580+0000 7efc1fe00640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efc0406c8f0 0x7efc0406edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efc2806e920 tx=0x7efc10009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.582+0000 7efc25600640 1 -- 172.21.15.131:0/531199860 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc1805a390 con 0x7efc280b87c0 2024-04-28T12:38:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.783+0000 7efc2f400640 1 -- 172.21.15.131:0/531199860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc280d3680 con 0x7efc280b87c0 2024-04-28T12:38:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.784+0000 7efc25600640 1 -- 172.21.15.131:0/531199860 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7efc18059c90 con 0x7efc280b87c0 2024-04-28T12:38:33.780 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.787+0000 7efc2f400640 1 -- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efc0406c8f0 msgr2=0x7efc0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.787+0000 7efc2f400640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efc0406c8f0 0x7efc0406edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efc2806e920 tx=0x7efc10009040 comp rx=0 tx=0).stop 2024-04-28T12:38:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.787+0000 7efc2f400640 1 -- 172.21.15.131:0/531199860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc280b87c0 msgr2=0x7efc280cb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.787+0000 7efc2f400640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc280b87c0 0x7efc280cb140 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efc1800d5f0 tx=0x7efc1800dab0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.787+0000 7efc2f400640 1 -- 172.21.15.131:0/531199860 shutdown_connections 2024-04-28T12:38:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.787+0000 7efc2f400640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7efc0406c8f0 0x7efc0406edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.787+0000 7efc2f400640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc2815f730 0x7efc281f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.787+0000 7efc2f400640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc2815f350 0x7efc2809ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.787+0000 7efc2f400640 1 --2- 172.21.15.131:0/531199860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc280b87c0 0x7efc280cb140 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:33.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.787+0000 7efc2f400640 1 -- 172.21.15.131:0/531199860 >> 172.21.15.131:0/531199860 conn(0x7efc280d4fb0 msgr2=0x7efc280d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:33.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.787+0000 7efc2f400640 1 -- 172.21.15.131:0/531199860 shutdown_connections 2024-04-28T12:38:33.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:33.787+0000 7efc2f400640 1 -- 172.21.15.131:0/531199860 wait complete. 2024-04-28T12:38:33.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.004+0000 7f6949e00640 1 Processor -- start 2024-04-28T12:38:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.004+0000 7f6949e00640 1 -- start start 2024-04-28T12:38:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6949e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f69440d3830 0x7f69440d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6949e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69440b3cb0 0x7f69440b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6949e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69440d2d70 con 0x7f694415f180 2024-04-28T12:38:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6949e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f694415f560 con 0x7f69440d3830 2024-04-28T12:38:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6949e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f69440d06e0 con 0x7f69440b3cb0 2024-04-28T12:38:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6943e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f69440d3830 0x7f69440d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6943e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f69440d3830 0x7f69440d56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37480/0 (socket says 172.21.15.131:37480) 2024-04-28T12:38:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6943e00640 1 -- 172.21.15.131:0/1836117198 learned_addr learned my addr 172.21.15.131:0/1836117198 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6949400640 1 --2- 172.21.15.131:0/1836117198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69440b3cb0 0x7f69440b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6943e00640 1 -- 172.21.15.131:0/1836117198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69440b3cb0 msgr2=0x7f69440b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6943e00640 1 --2- 172.21.15.131:0/1836117198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69440b3cb0 0x7f69440b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6943e00640 1 -- 172.21.15.131:0/1836117198 >> v1:172.21.15.157:6789/0 conn(0x7f694415f180 legacy=0x7f69440d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.006+0000 7f6943e00640 1 -- 172.21.15.131:0/1836117198 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69440d09c0 con 0x7f69440d3830 2024-04-28T12:38:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.007+0000 7f6943e00640 1 --2- 172.21.15.131:0/1836117198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69440d3830 0x7f69440d56f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6930009870 tx=0x7f693002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328884a877594957 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.008+0000 7f6943400640 1 -- 172.21.15.131:0/1836117198 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930037070 con 0x7f69440d3830 2024-04-28T12:38:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.008+0000 7f6943400640 1 -- 172.21.15.131:0/1836117198 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6930032440 con 0x7f69440d3830 2024-04-28T12:38:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.008+0000 7f6943400640 1 -- 172.21.15.131:0/1836117198 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930032810 con 0x7f69440d3830 2024-04-28T12:38:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.008+0000 7f6949e00640 1 -- 172.21.15.131:0/1836117198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69440d3830 msgr2=0x7f69440d56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.008+0000 7f6949e00640 1 --2- 172.21.15.131:0/1836117198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69440d3830 0x7f69440d56f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6930009870 tx=0x7f693002a210 comp rx=0 tx=0).stop 2024-04-28T12:38:34.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.008+0000 7f6949e00640 1 -- 172.21.15.131:0/1836117198 shutdown_connections 2024-04-28T12:38:34.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.008+0000 7f6949e00640 1 --2- 172.21.15.131:0/1836117198 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69440b3cb0 0x7f69440b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.008+0000 7f6949e00640 1 --2- 172.21.15.131:0/1836117198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69440d3830 0x7f69440d56f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.008+0000 7f6949e00640 1 -- 172.21.15.131:0/1836117198 >> 172.21.15.131:0/1836117198 conn(0x7f69440d4bd0 msgr2=0x7f69440d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:34.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.009+0000 7f6949e00640 1 -- 172.21.15.131:0/1836117198 shutdown_connections 2024-04-28T12:38:34.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.009+0000 7f6949e00640 1 -- 172.21.15.131:0/1836117198 wait complete. 2024-04-28T12:38:34.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.010+0000 7f6949e00640 1 Processor -- start 2024-04-28T12:38:34.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.010+0000 7f6949e00640 1 -- start start 2024-04-28T12:38:34.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.011+0000 7f6949e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69440b3cb0 0x7f69440c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.011+0000 7f6949e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69440d3830 0x7f69440b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.011+0000 7f6949e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f694415f180 0x7f69441533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.011+0000 7f6949e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f69440d11c0 con 0x7f69440d3830 2024-04-28T12:38:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.011+0000 7f6949e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f69440d2a40 con 0x7f694415f180 2024-04-28T12:38:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.011+0000 7f6949e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f69440c8cc0 con 0x7f69440b3cb0 2024-04-28T12:38:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.012+0000 7f6943e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69440d3830 0x7f69440b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.012+0000 7f6943e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69440d3830 0x7f69440b8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37496/0 (socket says 172.21.15.131:37496) 2024-04-28T12:38:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.012+0000 7f6943e00640 1 -- 172.21.15.131:0/1170660358 learned_addr learned my addr 172.21.15.131:0/1170660358 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.012+0000 7f6943e00640 1 -- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69440b3cb0 msgr2=0x7f69440c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.012+0000 7f6943e00640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69440b3cb0 0x7f69440c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.012+0000 7f6943e00640 1 -- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f694415f180 msgr2=0x7f69441533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.012+0000 7f6949400640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f694415f180 0x7f69441533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.012+0000 7f6943e00640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f694415f180 0x7f69441533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.012+0000 7f6943e00640 1 -- 172.21.15.131:0/1170660358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69440c8fa0 con 0x7f69440d3830 2024-04-28T12:38:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.012+0000 7f6943e00640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69440d3830 0x7f69440b8040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6930000c00 tx=0x7f69300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.013+0000 7f6940c00640 1 -- 172.21.15.131:0/1170660358 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930037040 con 0x7f69440d3830 2024-04-28T12:38:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.013+0000 7f6940c00640 1 -- 172.21.15.131:0/1170660358 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6930042070 con 0x7f69440d3830 2024-04-28T12:38:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.013+0000 7f6940c00640 1 -- 172.21.15.131:0/1170660358 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930030bb0 con 0x7f69440d3830 2024-04-28T12:38:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.013+0000 7f6949e00640 1 -- 172.21.15.131:0/1170660358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69440c9280 con 0x7f69440d3830 2024-04-28T12:38:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.013+0000 7f6949e00640 1 -- 172.21.15.131:0/1170660358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69440c9840 con 0x7f69440d3830 2024-04-28T12:38:34.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.014+0000 7f6949e00640 1 -- 172.21.15.131:0/1170660358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69440506c0 con 0x7f69440d3830 2024-04-28T12:38:34.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.015+0000 7f6940c00640 1 -- 172.21.15.131:0/1170660358 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6930047460 con 0x7f69440d3830 2024-04-28T12:38:34.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.016+0000 7f6940c00640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f692406c8f0 0x7f692406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.016+0000 7f6940c00640 1 -- 172.21.15.131:0/1170660358 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f69300acc40 con 0x7f69440d3830 2024-04-28T12:38:34.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.016+0000 7f6948a00640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f692406c8f0 0x7f692406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.018+0000 7f6948a00640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f692406c8f0 0x7f692406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6938006fd0 tx=0x7f6938006bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:34.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.020+0000 7f6940c00640 1 -- 172.21.15.131:0/1170660358 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6930045360 con 0x7f69440d3830 2024-04-28T12:38:34.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.218+0000 7f6949e00640 1 -- 172.21.15.131:0/1170660358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f694415e6c0 con 0x7f69440d3830 2024-04-28T12:38:34.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.219+0000 7f6940c00640 1 -- 172.21.15.131:0/1170660358 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f693007abd0 con 0x7f69440d3830 2024-04-28T12:38:34.216 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.222+0000 7f6949e00640 1 -- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f692406c8f0 msgr2=0x7f692406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.222+0000 7f6949e00640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f692406c8f0 0x7f692406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6938006fd0 tx=0x7f6938006bc0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.222+0000 7f6949e00640 1 -- 172.21.15.131:0/1170660358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69440d3830 msgr2=0x7f69440b8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.222+0000 7f6949e00640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69440d3830 0x7f69440b8040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6930000c00 tx=0x7f69300046c0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.222+0000 7f6949e00640 1 -- 172.21.15.131:0/1170660358 shutdown_connections 2024-04-28T12:38:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.222+0000 7f6949e00640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f692406c8f0 0x7f692406edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.222+0000 7f6949e00640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f694415f180 0x7f69441533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.222+0000 7f6949e00640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69440d3830 0x7f69440b8040 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.222+0000 7f6949e00640 1 --2- 172.21.15.131:0/1170660358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69440b3cb0 0x7f69440c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.222+0000 7f6949e00640 1 -- 172.21.15.131:0/1170660358 >> 172.21.15.131:0/1170660358 conn(0x7f69440d4bd0 msgr2=0x7f694415d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:34.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.222+0000 7f6949e00640 1 -- 172.21.15.131:0/1170660358 shutdown_connections 2024-04-28T12:38:34.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.222+0000 7f6949e00640 1 -- 172.21.15.131:0/1170660358 wait complete. 2024-04-28T12:38:34.238 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:34.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.444+0000 7f96b0e00640 1 Processor -- start 2024-04-28T12:38:34.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.444+0000 7f96b0e00640 1 -- start start 2024-04-28T12:38:34.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.446+0000 7f96b0e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96ac072a90 0x7f96ac0cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.446+0000 7f96b0e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f96ac15d5e0 0x7f96ac0cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.446+0000 7f96b0e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96ac146160 con 0x7f96ac145d80 2024-04-28T12:38:34.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.446+0000 7f96b0e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f96ac06dfb0 con 0x7f96ac15d5e0 2024-04-28T12:38:34.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.446+0000 7f96b0e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96ac074180 con 0x7f96ac072a90 2024-04-28T12:38:34.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.446+0000 7f96a9400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96ac072a90 0x7f96ac0cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.447+0000 7f96a9400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96ac072a90 0x7f96ac0cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:49354/0 (socket says 172.21.15.131:49354) 2024-04-28T12:38:34.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.447+0000 7f96a9400640 1 -- 172.21.15.131:0/3544655057 learned_addr learned my addr 172.21.15.131:0/3544655057 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:34.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.447+0000 7f96a9e00640 1 --1- 172.21.15.131:0/3544655057 >> v1:172.21.15.157:6789/0 conn(0x7f96ac145d80 0x7f96ac072380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.131:51088/0 (socket says 172.21.15.131:51088) 2024-04-28T12:38:34.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.447+0000 7f96aa800640 1 --2- 172.21.15.131:0/3544655057 >> v2:172.21.15.157:3300/0 conn(0x7f96ac15d5e0 0x7f96ac0cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.447+0000 7f96a8a00640 1 -- 172.21.15.131:0/3544655057 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2320699468 0 0) 0x7f96ac146160 con 0x7f96ac145d80 2024-04-28T12:38:34.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.447+0000 7f96a8a00640 1 -- 172.21.15.131:0/3544655057 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9690003520 con 0x7f96ac145d80 2024-04-28T12:38:34.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.447+0000 7f96a9400640 1 -- 172.21.15.131:0/3544655057 >> v2:172.21.15.157:3300/0 conn(0x7f96ac15d5e0 msgr2=0x7f96ac0cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:34.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.447+0000 7f96a9400640 1 --2- 172.21.15.131:0/3544655057 >> v2:172.21.15.157:3300/0 conn(0x7f96ac15d5e0 0x7f96ac0cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.447+0000 7f96a9400640 1 -- 172.21.15.131:0/3544655057 >> v1:172.21.15.157:6789/0 conn(0x7f96ac145d80 legacy=0x7f96ac072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.447+0000 7f96a9400640 1 -- 172.21.15.131:0/3544655057 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac0d0710 con 0x7f96ac072a90 2024-04-28T12:38:34.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.447+0000 7f96aa800640 1 --2- 172.21.15.131:0/3544655057 >> v2:172.21.15.157:3300/0 conn(0x7f96ac15d5e0 0x7f96ac0cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:38:34.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.449+0000 7f96a9400640 1 --2- 172.21.15.131:0/3544655057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96ac072a90 0x7f96ac0cf460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9698009870 tx=0x7f969802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d24e3b4cd2fd0de server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:34.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.449+0000 7f96a8a00640 1 -- 172.21.15.131:0/3544655057 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698037070 con 0x7f96ac072a90 2024-04-28T12:38:34.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.449+0000 7f96a8a00640 1 -- 172.21.15.131:0/3544655057 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9698032440 con 0x7f96ac072a90 2024-04-28T12:38:34.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.449+0000 7f96a8a00640 1 -- 172.21.15.131:0/3544655057 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698032810 con 0x7f96ac072a90 2024-04-28T12:38:34.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.450+0000 7f96b0e00640 1 -- 172.21.15.131:0/3544655057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96ac072a90 msgr2=0x7f96ac0cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.450+0000 7f96b0e00640 1 --2- 172.21.15.131:0/3544655057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96ac072a90 0x7f96ac0cf460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9698009870 tx=0x7f969802a270 comp rx=0 tx=0).stop 2024-04-28T12:38:34.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.450+0000 7f96b0e00640 1 -- 172.21.15.131:0/3544655057 shutdown_connections 2024-04-28T12:38:34.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.450+0000 7f96b0e00640 1 --2- 172.21.15.131:0/3544655057 >> v2:172.21.15.157:3300/0 conn(0x7f96ac15d5e0 0x7f96ac0cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.450+0000 7f96b0e00640 1 --2- 172.21.15.131:0/3544655057 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96ac072a90 0x7f96ac0cf460 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.450+0000 7f96b0e00640 1 -- 172.21.15.131:0/3544655057 >> 172.21.15.131:0/3544655057 conn(0x7f96ac0b2900 msgr2=0x7f96ac0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:34.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.450+0000 7f96b0e00640 1 -- 172.21.15.131:0/3544655057 shutdown_connections 2024-04-28T12:38:34.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.450+0000 7f96b0e00640 1 -- 172.21.15.131:0/3544655057 wait complete. 2024-04-28T12:38:34.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.452+0000 7f96b0e00640 1 Processor -- start 2024-04-28T12:38:34.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.452+0000 7f96b0e00640 1 -- start start 2024-04-28T12:38:34.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.453+0000 7f96b0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac072a90 0x7f96ac146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.453+0000 7f96b0e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96ac145d80 0x7f96ac14a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.453+0000 7f96b0e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ac15d5e0 0x7f96ac14aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.453+0000 7f96b0e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac0d09f0 con 0x7f96ac15d5e0 2024-04-28T12:38:34.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.453+0000 7f96b0e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac06e530 con 0x7f96ac072a90 2024-04-28T12:38:34.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.453+0000 7f96b0e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96ac1468c0 con 0x7f96ac145d80 2024-04-28T12:38:34.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.453+0000 7f96aa800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ac15d5e0 0x7f96ac14aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.453+0000 7f96aa800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ac15d5e0 0x7f96ac14aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37500/0 (socket says 172.21.15.131:37500) 2024-04-28T12:38:34.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.453+0000 7f96aa800640 1 -- 172.21.15.131:0/4087795741 learned_addr learned my addr 172.21.15.131:0/4087795741 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:34.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.453+0000 7f96a9400640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96ac145d80 0x7f96ac14a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.453+0000 7f96a9e00640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac072a90 0x7f96ac146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.454+0000 7f96aa800640 1 -- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96ac145d80 msgr2=0x7f96ac14a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.454+0000 7f96aa800640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96ac145d80 0x7f96ac14a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.454+0000 7f96aa800640 1 -- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac072a90 msgr2=0x7f96ac146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.454+0000 7f96aa800640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac072a90 0x7f96ac146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.454+0000 7f96aa800640 1 -- 172.21.15.131:0/4087795741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac1e8630 con 0x7f96ac15d5e0 2024-04-28T12:38:34.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.454+0000 7f96a9e00640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac072a90 0x7f96ac146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.454+0000 7f96aa800640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ac15d5e0 0x7f96ac14aa40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f96a400ef90 tx=0x7f96a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.454+0000 7f96a2000640 1 -- 172.21.15.131:0/4087795741 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4018020 con 0x7f96ac15d5e0 2024-04-28T12:38:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.454+0000 7f96a2000640 1 -- 172.21.15.131:0/4087795741 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a4014480 con 0x7f96ac15d5e0 2024-04-28T12:38:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.454+0000 7f96b0e00640 1 -- 172.21.15.131:0/4087795741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ac149460 con 0x7f96ac15d5e0 2024-04-28T12:38:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.454+0000 7f96a2000640 1 -- 172.21.15.131:0/4087795741 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4014850 con 0x7f96ac15d5e0 2024-04-28T12:38:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.454+0000 7f96b0e00640 1 -- 172.21.15.131:0/4087795741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ac149a20 con 0x7f96ac15d5e0 2024-04-28T12:38:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.456+0000 7f96b0e00640 1 -- 172.21.15.131:0/4087795741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96ac069350 con 0x7f96ac15d5e0 2024-04-28T12:38:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.456+0000 7f96a2000640 1 -- 172.21.15.131:0/4087795741 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96a4005d10 con 0x7f96ac15d5e0 2024-04-28T12:38:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.457+0000 7f96a2000640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f968c06c8f0 0x7f968c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.457+0000 7f96a2000640 1 -- 172.21.15.131:0/4087795741 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f96a408cd30 con 0x7f96ac15d5e0 2024-04-28T12:38:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.458+0000 7f96a9e00640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f968c06c8f0 0x7f968c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.460+0000 7f96a9e00640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f968c06c8f0 0x7f968c06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f969c0011e0 tx=0x7f969c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.462+0000 7f96a2000640 1 -- 172.21.15.131:0/4087795741 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96a4022030 con 0x7f96ac15d5e0 2024-04-28T12:38:34.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.658+0000 7f96b0e00640 1 -- 172.21.15.131:0/4087795741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f96ac072640 con 0x7f96ac15d5e0 2024-04-28T12:38:34.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.659+0000 7f96a2000640 1 -- 172.21.15.131:0/4087795741 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f96a4010040 con 0x7f96ac15d5e0 2024-04-28T12:38:34.656 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:34.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.662+0000 7f96b0e00640 1 -- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f968c06c8f0 msgr2=0x7f968c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.662+0000 7f96b0e00640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f968c06c8f0 0x7f968c06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f969c0011e0 tx=0x7f969c009040 comp rx=0 tx=0).stop 2024-04-28T12:38:34.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.662+0000 7f96b0e00640 1 -- 172.21.15.131:0/4087795741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ac15d5e0 msgr2=0x7f96ac14aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.662+0000 7f96b0e00640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ac15d5e0 0x7f96ac14aa40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f96a400ef90 tx=0x7f96a400c490 comp rx=0 tx=0).stop 2024-04-28T12:38:34.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.662+0000 7f96b0e00640 1 -- 172.21.15.131:0/4087795741 shutdown_connections 2024-04-28T12:38:34.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.662+0000 7f96b0e00640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f968c06c8f0 0x7f968c06edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.662+0000 7f96b0e00640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ac15d5e0 0x7f96ac14aa40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.662+0000 7f96b0e00640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96ac145d80 0x7f96ac14a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.662+0000 7f96b0e00640 1 --2- 172.21.15.131:0/4087795741 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96ac072a90 0x7f96ac146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.662+0000 7f96b0e00640 1 -- 172.21.15.131:0/4087795741 >> 172.21.15.131:0/4087795741 conn(0x7f96ac0b2900 msgr2=0x7f96ac145020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:34.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.662+0000 7f96b0e00640 1 -- 172.21.15.131:0/4087795741 shutdown_connections 2024-04-28T12:38:34.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.662+0000 7f96b0e00640 1 -- 172.21.15.131:0/4087795741 wait complete. 2024-04-28T12:38:34.678 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:34.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.884+0000 7f4587e00640 1 Processor -- start 2024-04-28T12:38:34.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.884+0000 7f4587e00640 1 -- start start 2024-04-28T12:38:34.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.885+0000 7f4587e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458815b980 0x7f458806ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.885+0000 7f4587e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f45880b4d20 0x7f458806f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.886+0000 7f4587e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458815c2a0 con 0x7f458815bd60 2024-04-28T12:38:34.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.886+0000 7f4587e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f458815d060 con 0x7f45880b4d20 2024-04-28T12:38:34.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.886+0000 7f4587e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f458815c580 con 0x7f458815b980 2024-04-28T12:38:34.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.886+0000 7f4587400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f45880b4d20 0x7f458806f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.886+0000 7f4587400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f45880b4d20 0x7f458806f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37504/0 (socket says 172.21.15.131:37504) 2024-04-28T12:38:34.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.886+0000 7f4587400640 1 -- 172.21.15.131:0/2699217420 learned_addr learned my addr 172.21.15.131:0/2699217420 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:34.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.886+0000 7f4586a00640 1 --2- 172.21.15.131:0/2699217420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458815b980 0x7f458806ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.886+0000 7f4587400640 1 -- 172.21.15.131:0/2699217420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458815b980 msgr2=0x7f458806ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:34.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.886+0000 7f4587400640 1 --2- 172.21.15.131:0/2699217420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458815b980 0x7f458806ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.886+0000 7f4587400640 1 -- 172.21.15.131:0/2699217420 >> v1:172.21.15.157:6789/0 conn(0x7f458815bd60 legacy=0x7f45880b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.887+0000 7f4587400640 1 -- 172.21.15.131:0/2699217420 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458815c860 con 0x7f45880b4d20 2024-04-28T12:38:34.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.888+0000 7f4587400640 1 --2- 172.21.15.131:0/2699217420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45880b4d20 0x7f458806f1d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4580009e50 tx=0x7f458002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2882be229d174a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:34.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.888+0000 7f4586000640 1 -- 172.21.15.131:0/2699217420 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580036070 con 0x7f45880b4d20 2024-04-28T12:38:34.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.888+0000 7f4586000640 1 -- 172.21.15.131:0/2699217420 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4580031600 con 0x7f45880b4d20 2024-04-28T12:38:34.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.888+0000 7f4586000640 1 -- 172.21.15.131:0/2699217420 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45800319d0 con 0x7f45880b4d20 2024-04-28T12:38:34.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.889+0000 7f4587e00640 1 -- 172.21.15.131:0/2699217420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45880b4d20 msgr2=0x7f458806f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.889+0000 7f4587e00640 1 --2- 172.21.15.131:0/2699217420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45880b4d20 0x7f458806f1d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4580009e50 tx=0x7f458002a710 comp rx=0 tx=0).stop 2024-04-28T12:38:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.889+0000 7f4587e00640 1 -- 172.21.15.131:0/2699217420 shutdown_connections 2024-04-28T12:38:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.889+0000 7f4587e00640 1 --2- 172.21.15.131:0/2699217420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45880b4d20 0x7f458806f1d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.889+0000 7f4587e00640 1 --2- 172.21.15.131:0/2699217420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f458815b980 0x7f458806ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.889+0000 7f4587e00640 1 -- 172.21.15.131:0/2699217420 >> 172.21.15.131:0/2699217420 conn(0x7f4588068090 msgr2=0x7f45880684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.889+0000 7f4587e00640 1 -- 172.21.15.131:0/2699217420 shutdown_connections 2024-04-28T12:38:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.889+0000 7f4587e00640 1 -- 172.21.15.131:0/2699217420 wait complete. 2024-04-28T12:38:34.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.891+0000 7f4587e00640 1 Processor -- start 2024-04-28T12:38:34.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.891+0000 7f4587e00640 1 -- start start 2024-04-28T12:38:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f4587e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45880b4d20 0x7f4588154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f4587e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f458815b980 0x7f45881e4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f4587e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f458815bd60 0x7f45881e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f4587e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f458815f6b0 con 0x7f458815b980 2024-04-28T12:38:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f4587e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f45880b52b0 con 0x7f458815bd60 2024-04-28T12:38:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f4587e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f458815f830 con 0x7f45880b4d20 2024-04-28T12:38:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f457fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f458815b980 0x7f45881e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f457fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f458815b980 0x7f45881e4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37518/0 (socket says 172.21.15.131:37518) 2024-04-28T12:38:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f457fe00640 1 -- 172.21.15.131:0/2199686982 learned_addr learned my addr 172.21.15.131:0/2199686982 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f4587400640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f458815bd60 0x7f45881e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f4586a00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45880b4d20 0x7f4588154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f457fe00640 1 -- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45880b4d20 msgr2=0x7f4588154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f457fe00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45880b4d20 0x7f4588154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f457fe00640 1 -- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f458815bd60 msgr2=0x7f45881e6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f457fe00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f458815bd60 0x7f45881e6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.892+0000 7f457fe00640 1 -- 172.21.15.131:0/2199686982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45881e6330 con 0x7f458815b980 2024-04-28T12:38:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.893+0000 7f457fe00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f458815b980 0x7f45881e4920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f457000c3c0 tx=0x7f457000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.893+0000 7f457ea00640 1 -- 172.21.15.131:0/2199686982 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45700052d0 con 0x7f458815b980 2024-04-28T12:38:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.893+0000 7f457ea00640 1 -- 172.21.15.131:0/2199686982 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4570005490 con 0x7f458815b980 2024-04-28T12:38:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.893+0000 7f457ea00640 1 -- 172.21.15.131:0/2199686982 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4570007260 con 0x7f458815b980 2024-04-28T12:38:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.893+0000 7f4587e00640 1 -- 172.21.15.131:0/2199686982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45881e5290 con 0x7f458815b980 2024-04-28T12:38:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.893+0000 7f4587e00640 1 -- 172.21.15.131:0/2199686982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45881e5850 con 0x7f458815b980 2024-04-28T12:38:34.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.894+0000 7f4587e00640 1 -- 172.21.15.131:0/2199686982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45880506c0 con 0x7f458815b980 2024-04-28T12:38:34.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.895+0000 7f457ea00640 1 -- 172.21.15.131:0/2199686982 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4570007400 con 0x7f458815b980 2024-04-28T12:38:34.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.896+0000 7f457ea00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f456406c8f0 0x7f456406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:34.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.896+0000 7f457ea00640 1 -- 172.21.15.131:0/2199686982 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f457008b490 con 0x7f458815b980 2024-04-28T12:38:34.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.897+0000 7f4586a00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f456406c8f0 0x7f456406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.899+0000 7f4586a00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f456406c8f0 0x7f456406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4578006fd0 tx=0x7f4578008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:34.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:34.901+0000 7f457ea00640 1 -- 172.21.15.131:0/2199686982 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4570059420 con 0x7f458815b980 2024-04-28T12:38:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.100+0000 7f4587e00640 1 -- 172.21.15.131:0/2199686982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f45880b4250 con 0x7f458815b980 2024-04-28T12:38:35.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.101+0000 7f457ea00640 1 -- 172.21.15.131:0/2199686982 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f4570058d20 con 0x7f458815b980 2024-04-28T12:38:35.099 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.103+0000 7f4587e00640 1 -- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f456406c8f0 msgr2=0x7f456406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.103+0000 7f4587e00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f456406c8f0 0x7f456406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4578006fd0 tx=0x7f4578008040 comp rx=0 tx=0).stop 2024-04-28T12:38:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.103+0000 7f4587e00640 1 -- 172.21.15.131:0/2199686982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f458815b980 msgr2=0x7f45881e4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.103+0000 7f4587e00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f458815b980 0x7f45881e4920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f457000c3c0 tx=0x7f457000c880 comp rx=0 tx=0).stop 2024-04-28T12:38:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.104+0000 7f4587e00640 1 -- 172.21.15.131:0/2199686982 shutdown_connections 2024-04-28T12:38:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.104+0000 7f4587e00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f456406c8f0 0x7f456406edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.104+0000 7f4587e00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f458815bd60 0x7f45881e6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.104+0000 7f4587e00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f458815b980 0x7f45881e4920 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.104+0000 7f4587e00640 1 --2- 172.21.15.131:0/2199686982 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45880b4d20 0x7f4588154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.104+0000 7f4587e00640 1 -- 172.21.15.131:0/2199686982 >> 172.21.15.131:0/2199686982 conn(0x7f4588068090 msgr2=0x7f45881573a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.104+0000 7f4587e00640 1 -- 172.21.15.131:0/2199686982 shutdown_connections 2024-04-28T12:38:35.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.104+0000 7f4587e00640 1 -- 172.21.15.131:0/2199686982 wait complete. 2024-04-28T12:38:35.121 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T12:38:35.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.316+0000 7fcd5f200640 1 Processor -- start 2024-04-28T12:38:35.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.317+0000 7fcd5f200640 1 -- start start 2024-04-28T12:38:35.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.318+0000 7fcd5f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd600d1620 0x7fcd600d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:35.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.318+0000 7fcd5f200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd600d3810 0x7fcd600b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:35.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.318+0000 7fcd5f200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd600d0480 con 0x7fcd600d1a00 2024-04-28T12:38:35.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.318+0000 7fcd5f200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcd600d0ce0 con 0x7fcd600d3810 2024-04-28T12:38:35.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.318+0000 7fcd5f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd600d0720 con 0x7fcd600d1620 2024-04-28T12:38:35.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.319+0000 7fcd5e800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd600d3810 0x7fcd600b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:35.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.319+0000 7fcd5e800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd600d3810 0x7fcd600b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37528/0 (socket says 172.21.15.131:37528) 2024-04-28T12:38:35.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.319+0000 7fcd5e800640 1 -- 172.21.15.131:0/124528035 learned_addr learned my addr 172.21.15.131:0/124528035 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:35.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.319+0000 7fcd5de00640 1 --2- 172.21.15.131:0/124528035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd600d1620 0x7fcd600d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:35.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.319+0000 7fcd5e800640 1 -- 172.21.15.131:0/124528035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd600d1620 msgr2=0x7fcd600d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:35.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.319+0000 7fcd5e800640 1 --2- 172.21.15.131:0/124528035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd600d1620 0x7fcd600d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.319+0000 7fcd5e800640 1 -- 172.21.15.131:0/124528035 >> v1:172.21.15.157:6789/0 conn(0x7fcd600d1a00 legacy=0x7fcd600b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:35.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.319+0000 7fcd5e800640 1 -- 172.21.15.131:0/124528035 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd600d0a00 con 0x7fcd600d3810 2024-04-28T12:38:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.320+0000 7fcd5e800640 1 --2- 172.21.15.131:0/124528035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd600d3810 0x7fcd600b5280 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009870 tx=0x7fcd5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ebec136602acd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.321+0000 7fcd5ca00640 1 -- 172.21.15.131:0/124528035 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58037070 con 0x7fcd600d3810 2024-04-28T12:38:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.321+0000 7fcd5ca00640 1 -- 172.21.15.131:0/124528035 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd58032440 con 0x7fcd600d3810 2024-04-28T12:38:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.321+0000 7fcd5ca00640 1 -- 172.21.15.131:0/124528035 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58032810 con 0x7fcd600d3810 2024-04-28T12:38:35.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.321+0000 7fcd5f200640 1 -- 172.21.15.131:0/124528035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd600d3810 msgr2=0x7fcd600b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:35.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.321+0000 7fcd5f200640 1 --2- 172.21.15.131:0/124528035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd600d3810 0x7fcd600b5280 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009870 tx=0x7fcd5802a180 comp rx=0 tx=0).stop 2024-04-28T12:38:35.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.321+0000 7fcd5f200640 1 -- 172.21.15.131:0/124528035 shutdown_connections 2024-04-28T12:38:35.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.321+0000 7fcd5f200640 1 --2- 172.21.15.131:0/124528035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd600d3810 0x7fcd600b5280 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.321+0000 7fcd5f200640 1 --2- 172.21.15.131:0/124528035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd600d1620 0x7fcd600d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.321+0000 7fcd5f200640 1 -- 172.21.15.131:0/124528035 >> 172.21.15.131:0/124528035 conn(0x7fcd600d4c30 msgr2=0x7fcd600d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:35.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.321+0000 7fcd5f200640 1 -- 172.21.15.131:0/124528035 shutdown_connections 2024-04-28T12:38:35.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.321+0000 7fcd5f200640 1 -- 172.21.15.131:0/124528035 wait complete. 2024-04-28T12:38:35.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.323+0000 7fcd5f200640 1 Processor -- start 2024-04-28T12:38:35.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.323+0000 7fcd5f200640 1 -- start start 2024-04-28T12:38:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.324+0000 7fcd5f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd600d1620 0x7fcd60146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.324+0000 7fcd5f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd600d1a00 0x7fcd601e9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.324+0000 7fcd5f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd600d3810 0x7fcd600b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.324+0000 7fcd5f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcd600b57c0 con 0x7fcd600d1a00 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.324+0000 7fcd5f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd600d1e60 con 0x7fcd600d1620 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.324+0000 7fcd5f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd600b5940 con 0x7fcd600d3810 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.324+0000 7fcd5d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd600d1a00 0x7fcd601e9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.324+0000 7fcd5d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd600d1a00 0x7fcd601e9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37544/0 (socket says 172.21.15.131:37544) 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.324+0000 7fcd5d400640 1 -- 172.21.15.131:0/2546099972 learned_addr learned my addr 172.21.15.131:0/2546099972 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.325+0000 7fcd5d400640 1 -- 172.21.15.131:0/2546099972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd600d3810 msgr2=0x7fcd600b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.325+0000 7fcd5d400640 1 --2- 172.21.15.131:0/2546099972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd600d3810 0x7fcd600b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.325+0000 7fcd5d400640 1 -- 172.21.15.131:0/2546099972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd600d1620 msgr2=0x7fcd60146320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.325+0000 7fcd5d400640 1 --2- 172.21.15.131:0/2546099972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd600d1620 0x7fcd60146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.325+0000 7fcd5d400640 1 -- 172.21.15.131:0/2546099972 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd600b9180 con 0x7fcd600d1a00 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.325+0000 7fcd5d400640 1 --2- 172.21.15.131:0/2546099972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd600d1a00 0x7fcd601e9bb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800c7c0 tx=0x7fcd4800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:35.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.325+0000 7fcd56000640 1 -- 172.21.15.131:0/2546099972 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48016020 con 0x7fcd600d1a00 2024-04-28T12:38:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.325+0000 7fcd56000640 1 -- 172.21.15.131:0/2546099972 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4800aa50 con 0x7fcd600d1a00 2024-04-28T12:38:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.325+0000 7fcd56000640 1 -- 172.21.15.131:0/2546099972 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48012420 con 0x7fcd600d1a00 2024-04-28T12:38:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.325+0000 7fcd5f200640 1 -- 172.21.15.131:0/2546099972 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd600b86e0 con 0x7fcd600d1a00 2024-04-28T12:38:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.325+0000 7fcd5f200640 1 -- 172.21.15.131:0/2546099972 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd600b9700 con 0x7fcd600d1a00 2024-04-28T12:38:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.327+0000 7fcd5f200640 1 -- 172.21.15.131:0/2546099972 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd60069c20 con 0x7fcd600d1a00 2024-04-28T12:38:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.328+0000 7fcd56000640 1 -- 172.21.15.131:0/2546099972 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd48019020 con 0x7fcd600d1a00 2024-04-28T12:38:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.329+0000 7fcd56000640 1 --2- 172.21.15.131:0/2546099972 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcd3c06c920 0x7fcd3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.329+0000 7fcd56000640 1 -- 172.21.15.131:0/2546099972 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcd4808bc40 con 0x7fcd600d1a00 2024-04-28T12:38:35.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.330+0000 7fcd5de00640 1 --2- 172.21.15.131:0/2546099972 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcd3c06c920 0x7fcd3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:35.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.331+0000 7fcd5de00640 1 --2- 172.21.15.131:0/2546099972 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcd3c06c920 0x7fcd3c06ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcd50006a90 tx=0x7fcd50006a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.334+0000 7fcd56000640 1 -- 172.21.15.131:0/2546099972 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd60069c20 con 0x7fcd600d1a00 2024-04-28T12:38:35.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.549+0000 7fcd5f200640 1 -- 172.21.15.131:0/2546099972 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcd600b9900 con 0x7fcd600d1a00 2024-04-28T12:38:35.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.550+0000 7fcd56000640 1 -- 172.21.15.131:0/2546099972 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fcd600b9900 con 0x7fcd600d1a00 2024-04-28T12:38:35.547 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-04-28T12:38:35.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.552+0000 7fcd5f200640 1 -- 172.21.15.131:0/2546099972 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcd3c06c920 msgr2=0x7fcd3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:35.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.552+0000 7fcd5f200640 1 --2- 172.21.15.131:0/2546099972 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcd3c06c920 0x7fcd3c06ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcd50006a90 tx=0x7fcd50006a20 comp rx=0 tx=0).stop 2024-04-28T12:38:35.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.553+0000 7fcd5f200640 1 -- 172.21.15.131:0/2546099972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd600d1a00 msgr2=0x7fcd601e9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:35.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.553+0000 7fcd5f200640 1 --2- 172.21.15.131:0/2546099972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd600d1a00 0x7fcd601e9bb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800c7c0 tx=0x7fcd4800cc80 comp rx=0 tx=0).stop 2024-04-28T12:38:35.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.553+0000 7fcd5f200640 1 -- 172.21.15.131:0/2546099972 shutdown_connections 2024-04-28T12:38:35.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.553+0000 7fcd5f200640 1 --2- 172.21.15.131:0/2546099972 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcd3c06c920 0x7fcd3c06ede0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.553+0000 7fcd5f200640 1 --2- 172.21.15.131:0/2546099972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd600d3810 0x7fcd600b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.553+0000 7fcd5f200640 1 --2- 172.21.15.131:0/2546099972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd600d1a00 0x7fcd601e9bb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.553+0000 7fcd5f200640 1 --2- 172.21.15.131:0/2546099972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd600d1620 0x7fcd60146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:35.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.553+0000 7fcd5f200640 1 -- 172.21.15.131:0/2546099972 >> 172.21.15.131:0/2546099972 conn(0x7fcd600d4c30 msgr2=0x7fcd6006c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:35.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.553+0000 7fcd5f200640 1 -- 172.21.15.131:0/2546099972 shutdown_connections 2024-04-28T12:38:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:35.553+0000 7fcd5f200640 1 -- 172.21.15.131:0/2546099972 wait complete. 2024-04-28T12:38:35.568 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-04-28T12:38:35.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:35.568 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-28T12:38:35.651 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:38:35.651 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:38:35.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:35.651 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-28T12:38:35.700 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:38:35.701 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:38:35.759 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:38:35.759 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:38:35.810 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:36.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.023+0000 7f5633200640 1 Processor -- start 2024-04-28T12:38:36.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.023+0000 7f5633200640 1 -- start start 2024-04-28T12:38:36.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.024+0000 7f5633200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5634145330 0x7f56340b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.025+0000 7f5633200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56341628e0 0x7f5634162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.025+0000 7f5633200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5634145710 con 0x7f56340af720 2024-04-28T12:38:36.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.025+0000 7f5633200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5634163280 con 0x7f5634145330 2024-04-28T12:38:36.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.025+0000 7f5633200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5634073dc0 con 0x7f56341628e0 2024-04-28T12:38:36.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.025+0000 7f5631e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5634145330 0x7f56340b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.025+0000 7f5632800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f56340af720 0x7f56340afbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47112/0 (socket says 172.21.15.131:47112) 2024-04-28T12:38:36.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.025+0000 7f5632800640 1 -- 172.21.15.131:0/2253241681 learned_addr learned my addr 172.21.15.131:0/2253241681 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:36.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.025+0000 7f5631400640 1 --2- 172.21.15.131:0/2253241681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56341628e0 0x7f5634162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.026+0000 7f5631e00640 1 -- 172.21.15.131:0/2253241681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56341628e0 msgr2=0x7f5634162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:36.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.026+0000 7f5631e00640 1 --2- 172.21.15.131:0/2253241681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56341628e0 0x7f5634162d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.026+0000 7f5631e00640 1 -- 172.21.15.131:0/2253241681 >> v1:172.21.15.131:6789/0 conn(0x7f56340af720 legacy=0x7f56340afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.026+0000 7f5630a00640 1 -- 172.21.15.131:0/2253241681 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 112134172 0 0) 0x7f5634145710 con 0x7f56340af720 2024-04-28T12:38:36.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.026+0000 7f5631e00640 1 -- 172.21.15.131:0/2253241681 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56340ae960 con 0x7f5634145330 2024-04-28T12:38:36.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.027+0000 7f5631e00640 1 --2- 172.21.15.131:0/2253241681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634145330 0x7f56340b3940 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f56240097a0 tx=0x7f562402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4938cc513bcfb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:36.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.027+0000 7f5630a00640 1 -- 172.21.15.131:0/2253241681 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624037070 con 0x7f5634145330 2024-04-28T12:38:36.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.027+0000 7f5630a00640 1 -- 172.21.15.131:0/2253241681 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5624032440 con 0x7f5634145330 2024-04-28T12:38:36.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.027+0000 7f5630a00640 1 -- 172.21.15.131:0/2253241681 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624032810 con 0x7f5634145330 2024-04-28T12:38:36.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.028+0000 7f5633200640 1 -- 172.21.15.131:0/2253241681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634145330 msgr2=0x7f56340b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.028+0000 7f5633200640 1 --2- 172.21.15.131:0/2253241681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634145330 0x7f56340b3940 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f56240097a0 tx=0x7f562402a180 comp rx=0 tx=0).stop 2024-04-28T12:38:36.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.028+0000 7f5633200640 1 -- 172.21.15.131:0/2253241681 shutdown_connections 2024-04-28T12:38:36.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.028+0000 7f5633200640 1 --2- 172.21.15.131:0/2253241681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f56341628e0 0x7f5634162d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.028+0000 7f5633200640 1 --2- 172.21.15.131:0/2253241681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634145330 0x7f56340b3940 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.028+0000 7f5633200640 1 -- 172.21.15.131:0/2253241681 >> 172.21.15.131:0/2253241681 conn(0x7f5634068090 msgr2=0x7f5634068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:36.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.028+0000 7f5633200640 1 -- 172.21.15.131:0/2253241681 shutdown_connections 2024-04-28T12:38:36.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.028+0000 7f5633200640 1 -- 172.21.15.131:0/2253241681 wait complete. 2024-04-28T12:38:36.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.029+0000 7f5633200640 1 Processor -- start 2024-04-28T12:38:36.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.030+0000 7f5633200640 1 -- start start 2024-04-28T12:38:36.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.030+0000 7f5633200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56340af720 0x7f563415c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.030+0000 7f5633200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5634145330 0x7f56340c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5633200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56341628e0 0x7f56340c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5633200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f56340af200 con 0x7f56341628e0 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5633200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f56340aef20 con 0x7f56340af720 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5633200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f56340b3e80 con 0x7f5634145330 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5632800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56341628e0 0x7f56340c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5632800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56341628e0 0x7f56340c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37574/0 (socket says 172.21.15.131:37574) 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5632800640 1 -- 172.21.15.131:0/141832768 learned_addr learned my addr 172.21.15.131:0/141832768 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5631400640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5634145330 0x7f56340c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5631e00640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56340af720 0x7f563415c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5632800640 1 -- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5634145330 msgr2=0x7f56340c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5632800640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5634145330 0x7f56340c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5632800640 1 -- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56340af720 msgr2=0x7f563415c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5632800640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56340af720 0x7f563415c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5632800640 1 -- 172.21.15.131:0/141832768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56340b2be0 con 0x7f56341628e0 2024-04-28T12:38:36.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.031+0000 7f5631400640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5634145330 0x7f56340c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:36.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.032+0000 7f5632800640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56341628e0 0x7f56340c86a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f562c00c7a0 tx=0x7f562c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:36.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.032+0000 7f562a000640 1 -- 172.21.15.131:0/141832768 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c018070 con 0x7f56341628e0 2024-04-28T12:38:36.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.032+0000 7f562a000640 1 -- 172.21.15.131:0/141832768 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562c00a640 con 0x7f56341628e0 2024-04-28T12:38:36.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.032+0000 7f562a000640 1 -- 172.21.15.131:0/141832768 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c00aab0 con 0x7f56341628e0 2024-04-28T12:38:36.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.032+0000 7f5633200640 1 -- 172.21.15.131:0/141832768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56340b2ec0 con 0x7f56341628e0 2024-04-28T12:38:36.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.032+0000 7f5633200640 1 -- 172.21.15.131:0/141832768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56340c5c30 con 0x7f56341628e0 2024-04-28T12:38:36.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.033+0000 7f5633200640 1 -- 172.21.15.131:0/141832768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5634069350 con 0x7f56341628e0 2024-04-28T12:38:36.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.034+0000 7f562a000640 1 -- 172.21.15.131:0/141832768 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f562c01b070 con 0x7f56341628e0 2024-04-28T12:38:36.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.035+0000 7f562a000640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f561406c8f0 0x7f561406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.035+0000 7f562a000640 1 -- 172.21.15.131:0/141832768 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f562c08b520 con 0x7f56341628e0 2024-04-28T12:38:36.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.036+0000 7f5631e00640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f561406c8f0 0x7f561406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.037+0000 7f5631e00640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f561406c8f0 0x7f561406edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5624034000 tx=0x7f562402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:36.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.039+0000 7f562a000640 1 -- 172.21.15.131:0/141832768 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f562c059540 con 0x7f56341628e0 2024-04-28T12:38:36.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.239+0000 7f5633200640 1 -- 172.21.15.131:0/141832768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5634071ed0 con 0x7f56341628e0 2024-04-28T12:38:36.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.242+0000 7f562a000640 1 -- 172.21.15.131:0/141832768 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7f562c058e40 con 0x7f56341628e0 2024-04-28T12:38:36.239 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:36.239 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:36.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.245+0000 7f5633200640 1 -- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f561406c8f0 msgr2=0x7f561406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.245+0000 7f5633200640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f561406c8f0 0x7f561406edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5624034000 tx=0x7f562402e000 comp rx=0 tx=0).stop 2024-04-28T12:38:36.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.245+0000 7f5633200640 1 -- 172.21.15.131:0/141832768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56341628e0 msgr2=0x7f56340c86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.245+0000 7f5633200640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56341628e0 0x7f56340c86a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f562c00c7a0 tx=0x7f562c00cc60 comp rx=0 tx=0).stop 2024-04-28T12:38:36.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.245+0000 7f5633200640 1 -- 172.21.15.131:0/141832768 shutdown_connections 2024-04-28T12:38:36.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.245+0000 7f5633200640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f561406c8f0 0x7f561406edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.245+0000 7f5633200640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56341628e0 0x7f56340c86a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.245+0000 7f5633200640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5634145330 0x7f56340c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.245+0000 7f5633200640 1 --2- 172.21.15.131:0/141832768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f56340af720 0x7f563415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.245+0000 7f5633200640 1 -- 172.21.15.131:0/141832768 >> 172.21.15.131:0/141832768 conn(0x7f5634068090 msgr2=0x7f563406f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.245+0000 7f5633200640 1 -- 172.21.15.131:0/141832768 shutdown_connections 2024-04-28T12:38:36.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.246+0000 7f5633200640 1 -- 172.21.15.131:0/141832768 wait complete. 2024-04-28T12:38:36.262 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:36.263 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:36.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.475+0000 7f0728000640 1 Processor -- start 2024-04-28T12:38:36.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.476+0000 7f0728000640 1 -- start start 2024-04-28T12:38:36.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.477+0000 7f0728000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07201458e0 0x7f07201604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.477+0000 7f0728000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f072006df40 0x7f07200b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.477+0000 7f0728000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f07200b52e0 0x7f072015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.477+0000 7f0728000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f07200b5730 con 0x7f07200b52e0 2024-04-28T12:38:36.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.477+0000 7f0728000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f07200b3d80 con 0x7f072006df40 2024-04-28T12:38:36.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.477+0000 7f0728000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f07200b4060 con 0x7f07201458e0 2024-04-28T12:38:36.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.478+0000 7f0726000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f07200b52e0 0x7f072015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.478+0000 7f0726000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f07200b52e0 0x7f072015c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37590/0 (socket says 172.21.15.131:37590) 2024-04-28T12:38:36.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.478+0000 7f0726000640 1 -- 172.21.15.131:0/2149319464 learned_addr learned my addr 172.21.15.131:0/2149319464 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:36.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.478+0000 7f0725600640 1 --2- 172.21.15.131:0/2149319464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07201458e0 0x7f07201604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.478+0000 7f0724c00640 1 --2- 172.21.15.131:0/2149319464 >> v2:172.21.15.157:3300/0 conn(0x7f072006df40 0x7f07200b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.478+0000 7f0726000640 1 -- 172.21.15.131:0/2149319464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07201458e0 msgr2=0x7f07201604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:36.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.478+0000 7f0726000640 1 --2- 172.21.15.131:0/2149319464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07201458e0 0x7f07201604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.478+0000 7f0726000640 1 -- 172.21.15.131:0/2149319464 >> v2:172.21.15.157:3300/0 conn(0x7f072006df40 msgr2=0x7f07200b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:36.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.478+0000 7f0726000640 1 --2- 172.21.15.131:0/2149319464 >> v2:172.21.15.157:3300/0 conn(0x7f072006df40 0x7f07200b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.478+0000 7f0726000640 1 -- 172.21.15.131:0/2149319464 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07200b4340 con 0x7f07200b52e0 2024-04-28T12:38:36.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.480+0000 7f0726000640 1 --2- 172.21.15.131:0/2149319464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07200b52e0 0x7f072015c4a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0720071bd0 tx=0x7f071802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c74f11c7148347 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:36.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.480+0000 7f071fe00640 1 -- 172.21.15.131:0/2149319464 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718036070 con 0x7f07200b52e0 2024-04-28T12:38:36.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.480+0000 7f071fe00640 1 -- 172.21.15.131:0/2149319464 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0718031600 con 0x7f07200b52e0 2024-04-28T12:38:36.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.480+0000 7f071fe00640 1 -- 172.21.15.131:0/2149319464 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07180319d0 con 0x7f07200b52e0 2024-04-28T12:38:36.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.481+0000 7f0728000640 1 -- 172.21.15.131:0/2149319464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07200b52e0 msgr2=0x7f072015c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.481+0000 7f0728000640 1 --2- 172.21.15.131:0/2149319464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07200b52e0 0x7f072015c4a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0720071bd0 tx=0x7f071802a610 comp rx=0 tx=0).stop 2024-04-28T12:38:36.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.481+0000 7f0728000640 1 -- 172.21.15.131:0/2149319464 shutdown_connections 2024-04-28T12:38:36.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.481+0000 7f0728000640 1 --2- 172.21.15.131:0/2149319464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f07200b52e0 0x7f072015c4a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.481+0000 7f0728000640 1 --2- 172.21.15.131:0/2149319464 >> v2:172.21.15.157:3300/0 conn(0x7f072006df40 0x7f07200b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.481+0000 7f0728000640 1 --2- 172.21.15.131:0/2149319464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07201458e0 0x7f07201604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.481+0000 7f0728000640 1 -- 172.21.15.131:0/2149319464 >> 172.21.15.131:0/2149319464 conn(0x7f07200b2900 msgr2=0x7f07200b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:36.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.481+0000 7f0728000640 1 -- 172.21.15.131:0/2149319464 shutdown_connections 2024-04-28T12:38:36.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.481+0000 7f0728000640 1 -- 172.21.15.131:0/2149319464 wait complete. 2024-04-28T12:38:36.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.482+0000 7f0728000640 1 Processor -- start 2024-04-28T12:38:36.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.483+0000 7f0728000640 1 -- start start 2024-04-28T12:38:36.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0728000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072006df40 0x7f072013dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0728000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07200b52e0 0x7f07200734a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0728000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07201458e0 0x7f072013d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0728000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f072015c9e0 con 0x7f072006df40 2024-04-28T12:38:36.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0728000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07200b4900 con 0x7f07201458e0 2024-04-28T12:38:36.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0728000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f072015cb60 con 0x7f07200b52e0 2024-04-28T12:38:36.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0725600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072006df40 0x7f072013dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0725600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072006df40 0x7f072013dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37596/0 (socket says 172.21.15.131:37596) 2024-04-28T12:38:36.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0725600640 1 -- 172.21.15.131:0/1780178386 learned_addr learned my addr 172.21.15.131:0/1780178386 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:36.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0724c00640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07200b52e0 0x7f07200734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0726000640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07201458e0 0x7f072013d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0725600640 1 -- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07200b52e0 msgr2=0x7f07200734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0725600640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07200b52e0 0x7f07200734a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0725600640 1 -- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07201458e0 msgr2=0x7f072013d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0725600640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07201458e0 0x7f072013d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0725600640 1 -- 172.21.15.131:0/1780178386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07200741d0 con 0x7f072006df40 2024-04-28T12:38:36.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.484+0000 7f0726000640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07201458e0 0x7f072013d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:36.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.485+0000 7f0725600640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072006df40 0x7f072013dff0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f071400d5f0 tx=0x7f071400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:36.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.485+0000 7f071d600640 1 -- 172.21.15.131:0/1780178386 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07140125f0 con 0x7f072006df40 2024-04-28T12:38:36.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.485+0000 7f071d600640 1 -- 172.21.15.131:0/1780178386 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0714019070 con 0x7f072006df40 2024-04-28T12:38:36.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.485+0000 7f071d600640 1 -- 172.21.15.131:0/1780178386 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07140129d0 con 0x7f072006df40 2024-04-28T12:38:36.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.485+0000 7f0728000640 1 -- 172.21.15.131:0/1780178386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0720143000 con 0x7f072006df40 2024-04-28T12:38:36.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.485+0000 7f0728000640 1 -- 172.21.15.131:0/1780178386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0720073ef0 con 0x7f072006df40 2024-04-28T12:38:36.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.486+0000 7f0728000640 1 -- 172.21.15.131:0/1780178386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0720069350 con 0x7f072006df40 2024-04-28T12:38:36.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.487+0000 7f071d600640 1 -- 172.21.15.131:0/1780178386 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0714012b70 con 0x7f072006df40 2024-04-28T12:38:36.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.488+0000 7f071d600640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f070006c8f0 0x7f070006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.488+0000 7f071d600640 1 -- 172.21.15.131:0/1780178386 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f071408c3b0 con 0x7f072006df40 2024-04-28T12:38:36.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.489+0000 7f0724c00640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f070006c8f0 0x7f070006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.490+0000 7f0724c00640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f070006c8f0 0x7f070006edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0720169030 tx=0x7f070c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:36.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.492+0000 7f071d600640 1 -- 172.21.15.131:0/1780178386 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f071405a3d0 con 0x7f072006df40 2024-04-28T12:38:36.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.690+0000 7f0728000640 1 -- 172.21.15.131:0/1780178386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f072006e5b0 con 0x7f072006df40 2024-04-28T12:38:36.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.691+0000 7f071d600640 1 -- 172.21.15.131:0/1780178386 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f0714059cd0 con 0x7f072006df40 2024-04-28T12:38:36.688 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.694+0000 7f0728000640 1 -- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f070006c8f0 msgr2=0x7f070006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.694+0000 7f0728000640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f070006c8f0 0x7f070006edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0720169030 tx=0x7f070c008040 comp rx=0 tx=0).stop 2024-04-28T12:38:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.694+0000 7f0728000640 1 -- 172.21.15.131:0/1780178386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072006df40 msgr2=0x7f072013dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.694+0000 7f0728000640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072006df40 0x7f072013dff0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f071400d5f0 tx=0x7f071400dab0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.694+0000 7f0728000640 1 -- 172.21.15.131:0/1780178386 shutdown_connections 2024-04-28T12:38:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.694+0000 7f0728000640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f070006c8f0 0x7f070006edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.694+0000 7f0728000640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07201458e0 0x7f072013d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.694+0000 7f0728000640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07200b52e0 0x7f07200734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.694+0000 7f0728000640 1 --2- 172.21.15.131:0/1780178386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f072006df40 0x7f072013dff0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.694+0000 7f0728000640 1 -- 172.21.15.131:0/1780178386 >> 172.21.15.131:0/1780178386 conn(0x7f07200b2900 msgr2=0x7f072006e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:36.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.694+0000 7f0728000640 1 -- 172.21.15.131:0/1780178386 shutdown_connections 2024-04-28T12:38:36.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.694+0000 7f0728000640 1 -- 172.21.15.131:0/1780178386 wait complete. 2024-04-28T12:38:36.710 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:36.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.922+0000 7f177ea00640 1 Processor -- start 2024-04-28T12:38:36.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.922+0000 7f177ea00640 1 -- start start 2024-04-28T12:38:36.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.923+0000 7f177ea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f177815a7e0 0x7f17780b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.924+0000 7f177ea00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17780b3900 con 0x7f177815d9c0 2024-04-28T12:38:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.924+0000 7f177ea00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17780b3b70 con 0x7f17780b5410 2024-04-28T12:38:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.924+0000 7f177ea00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f177815abc0 con 0x7f177815a7e0 2024-04-28T12:38:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.924+0000 7f177e000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f177815d9c0 0x7f177815e2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47116/0 (socket says 172.21.15.131:47116) 2024-04-28T12:38:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.924+0000 7f177e000640 1 -- 172.21.15.131:0/500453299 learned_addr learned my addr 172.21.15.131:0/500453299 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:36.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.924+0000 7f177d600640 1 --2- 172.21.15.131:0/500453299 >> v2:172.21.15.157:3300/0 conn(0x7f177815a7e0 0x7f17780b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.924+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2242704249 0 0) 0x7f17780b3900 con 0x7f177815d9c0 2024-04-28T12:38:36.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1760003520 con 0x7f177815d9c0 2024-04-28T12:38:36.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2555262968 0 0) 0x7f17780b3b70 con 0x7f17780b5410 2024-04-28T12:38:36.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17780b3900 con 0x7f17780b5410 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2524068859 0 0) 0x7f1760003520 con 0x7f177815d9c0 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 >> v2:172.21.15.157:3300/0 conn(0x7f177815a7e0 msgr2=0x7f17780b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 --2- 172.21.15.131:0/500453299 >> v2:172.21.15.157:3300/0 conn(0x7f177815a7e0 0x7f17780b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 >> v1:172.21.15.157:6789/0 conn(0x7f17780b5410 legacy=0x7f177815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17780b3e50 con 0x7f177815d9c0 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f17700019f0 con 0x7f177815d9c0 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 >> v1:172.21.15.131:6789/0 conn(0x7f177815d9c0 legacy=0x7f177815e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 --2- 172.21.15.131:0/500453299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1760003bb0 0x7f1760003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 --2- 172.21.15.131:0/500453299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1760004800 0x7f1760004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 --2- 172.21.15.131:0/500453299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17600051c0 0x7f176000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f177815abc0 con 0x7f1760003bb0 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f176000c060 con 0x7f17600051c0 2024-04-28T12:38:36.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f176000c260 con 0x7f1760004800 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.925+0000 7f177d600640 1 --2- 172.21.15.131:0/500453299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1760003bb0 0x7f1760003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.926+0000 7f177e000640 1 --2- 172.21.15.131:0/500453299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17600051c0 0x7f176000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.926+0000 7f177d600640 1 -- 172.21.15.131:0/500453299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1760004800 msgr2=0x7f1760004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.926+0000 7f177d600640 1 --2- 172.21.15.131:0/500453299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1760004800 0x7f1760004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.926+0000 7f177d600640 1 -- 172.21.15.131:0/500453299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17600051c0 msgr2=0x7f176000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.926+0000 7f177d600640 1 --2- 172.21.15.131:0/500453299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17600051c0 0x7f176000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.926+0000 7f177d600640 1 -- 172.21.15.131:0/500453299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17780b3e50 con 0x7f1760003bb0 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.926+0000 7f177e000640 1 --2- 172.21.15.131:0/500453299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17600051c0 0x7f176000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.926+0000 7f177e000640 1 -- 172.21.15.131:0/500453299 reap_dead start 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.927+0000 7f177d600640 1 --2- 172.21.15.131:0/500453299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1760003bb0 0x7f1760003fb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f176c009ac0 tx=0x7f1760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d900ad0712c0081a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.927+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c036070 con 0x7f1760003bb0 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.927+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f176c031400 con 0x7f1760003bb0 2024-04-28T12:38:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.927+0000 7f1777e00640 1 -- 172.21.15.131:0/500453299 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c009690 con 0x7f1760003bb0 2024-04-28T12:38:36.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.927+0000 7f177ea00640 1 -- 172.21.15.131:0/500453299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1760003bb0 msgr2=0x7f1760003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.927+0000 7f177ea00640 1 --2- 172.21.15.131:0/500453299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1760003bb0 0x7f1760003fb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f176c009ac0 tx=0x7f1760005830 comp rx=0 tx=0).stop 2024-04-28T12:38:36.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.928+0000 7f177ea00640 1 -- 172.21.15.131:0/500453299 shutdown_connections 2024-04-28T12:38:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.928+0000 7f177ea00640 1 --2- 172.21.15.131:0/500453299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1760003bb0 0x7f1760003fb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.928+0000 7f177ea00640 1 -- 172.21.15.131:0/500453299 >> 172.21.15.131:0/500453299 conn(0x7f1778068090 msgr2=0x7f17780684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.928+0000 7f177ea00640 1 -- 172.21.15.131:0/500453299 shutdown_connections 2024-04-28T12:38:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.928+0000 7f177ea00640 1 -- 172.21.15.131:0/500453299 wait complete. 2024-04-28T12:38:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.929+0000 7f177ea00640 1 Processor -- start 2024-04-28T12:38:36.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.930+0000 7f177ea00640 1 -- start start 2024-04-28T12:38:36.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.930+0000 7f177ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1760003bb0 0x7f17780cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.930+0000 7f177ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17780ca490 0x7f17781f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1778153010 0x7f17781534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177ea00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17780b5890 con 0x7f17780ca490 2024-04-28T12:38:36.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f177815e010 con 0x7f1778153010 2024-04-28T12:38:36.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f17780b5a10 con 0x7f1760003bb0 2024-04-28T12:38:36.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17780ca490 0x7f17781f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17780ca490 0x7f17781f54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37618/0 (socket says 172.21.15.131:37618) 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177cc00640 1 -- 172.21.15.131:0/1505021754 learned_addr learned my addr 172.21.15.131:0/1505021754 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177e000640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1778153010 0x7f17781534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177d600640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1760003bb0 0x7f17780cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177cc00640 1 -- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1760003bb0 msgr2=0x7f17780cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177cc00640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1760003bb0 0x7f17780cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177cc00640 1 -- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1778153010 msgr2=0x7f17781534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177cc00640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1778153010 0x7f17781534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.931+0000 7f177cc00640 1 -- 172.21.15.131:0/1505021754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17781f59f0 con 0x7f17780ca490 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.932+0000 7f177cc00640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17780ca490 0x7f17781f54b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f176400c4c0 tx=0x7f176400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.932+0000 7f1775600640 1 -- 172.21.15.131:0/1505021754 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640052d0 con 0x7f17780ca490 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.932+0000 7f1775600640 1 -- 172.21.15.131:0/1505021754 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1764005490 con 0x7f17780ca490 2024-04-28T12:38:36.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.932+0000 7f177ea00640 1 -- 172.21.15.131:0/1505021754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1778151770 con 0x7f17780ca490 2024-04-28T12:38:36.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.932+0000 7f1775600640 1 -- 172.21.15.131:0/1505021754 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764012870 con 0x7f17780ca490 2024-04-28T12:38:36.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.932+0000 7f177ea00640 1 -- 172.21.15.131:0/1505021754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1778071ee0 con 0x7f17780ca490 2024-04-28T12:38:36.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.933+0000 7f177ea00640 1 -- 172.21.15.131:0/1505021754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17780cc6f0 con 0x7f17780ca490 2024-04-28T12:38:36.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.935+0000 7f1775600640 1 -- 172.21.15.131:0/1505021754 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1764012a10 con 0x7f17780ca490 2024-04-28T12:38:36.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.936+0000 7f1775600640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f175806c8f0 0x7f175806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:36.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.936+0000 7f1775600640 1 -- 172.21.15.131:0/1505021754 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f176408bbb0 con 0x7f17780ca490 2024-04-28T12:38:36.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.937+0000 7f177d600640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f175806c8f0 0x7f175806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:36.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.937+0000 7f177d600640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f175806c8f0 0x7f175806edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f176c033000 tx=0x7f176c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:36.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:36.941+0000 7f1775600640 1 -- 172.21.15.131:0/1505021754 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17780cc6f0 con 0x7f17780ca490 2024-04-28T12:38:37.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.127+0000 7f177ea00640 1 -- 172.21.15.131:0/1505021754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f17780506c0 con 0x7f17780ca490 2024-04-28T12:38:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.128+0000 7f1775600640 1 -- 172.21.15.131:0/1505021754 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f17781f59f0 con 0x7f17780ca490 2024-04-28T12:38:37.125 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.130+0000 7f177ea00640 1 -- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f175806c8f0 msgr2=0x7f175806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.130+0000 7f177ea00640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f175806c8f0 0x7f175806edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f176c033000 tx=0x7f176c008030 comp rx=0 tx=0).stop 2024-04-28T12:38:37.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.130+0000 7f177ea00640 1 -- 172.21.15.131:0/1505021754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17780ca490 msgr2=0x7f17781f54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.130+0000 7f177ea00640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17780ca490 0x7f17781f54b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f176400c4c0 tx=0x7f176400c980 comp rx=0 tx=0).stop 2024-04-28T12:38:37.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.130+0000 7f177ea00640 1 -- 172.21.15.131:0/1505021754 shutdown_connections 2024-04-28T12:38:37.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.130+0000 7f177ea00640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f175806c8f0 0x7f175806edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.130+0000 7f177ea00640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1778153010 0x7f17781534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.130+0000 7f177ea00640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17780ca490 0x7f17781f54b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.130+0000 7f177ea00640 1 --2- 172.21.15.131:0/1505021754 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1760003bb0 0x7f17780cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.130+0000 7f177ea00640 1 -- 172.21.15.131:0/1505021754 >> 172.21.15.131:0/1505021754 conn(0x7f1778068090 msgr2=0x7f17780cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.130+0000 7f177ea00640 1 -- 172.21.15.131:0/1505021754 shutdown_connections 2024-04-28T12:38:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.130+0000 7f177ea00640 1 -- 172.21.15.131:0/1505021754 wait complete. 2024-04-28T12:38:37.145 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:37.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.349+0000 7fb615400640 1 Processor -- start 2024-04-28T12:38:37.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.349+0000 7fb615400640 1 -- start start 2024-04-28T12:38:37.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.350+0000 7fb615400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb610072a90 0x7fb6100cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:37.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.350+0000 7fb615400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb61015d5e0 0x7fb6100cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:37.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.350+0000 7fb615400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb610146160 con 0x7fb610145d80 2024-04-28T12:38:37.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.350+0000 7fb615400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb61006dfb0 con 0x7fb610072a90 2024-04-28T12:38:37.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.350+0000 7fb615400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb610074180 con 0x7fb61015d5e0 2024-04-28T12:38:37.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.351+0000 7fb607e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb610072a90 0x7fb6100cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:37.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.351+0000 7fb607e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb610072a90 0x7fb6100cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37630/0 (socket says 172.21.15.131:37630) 2024-04-28T12:38:37.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.351+0000 7fb607e00640 1 -- 172.21.15.131:0/3164467515 learned_addr learned my addr 172.21.15.131:0/3164467515 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:37.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.351+0000 7fb60f200640 1 --2- 172.21.15.131:0/3164467515 >> v2:172.21.15.157:3300/0 conn(0x7fb61015d5e0 0x7fb6100cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:37.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.351+0000 7fb607e00640 1 -- 172.21.15.131:0/3164467515 >> v2:172.21.15.157:3300/0 conn(0x7fb61015d5e0 msgr2=0x7fb6100cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:37.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.351+0000 7fb607e00640 1 --2- 172.21.15.131:0/3164467515 >> v2:172.21.15.157:3300/0 conn(0x7fb61015d5e0 0x7fb6100cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.351+0000 7fb607e00640 1 -- 172.21.15.131:0/3164467515 >> v1:172.21.15.157:6789/0 conn(0x7fb610145d80 legacy=0x7fb610072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:37.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.351+0000 7fb607e00640 1 -- 172.21.15.131:0/3164467515 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6100d0710 con 0x7fb610072a90 2024-04-28T12:38:37.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.353+0000 7fb607e00640 1 --2- 172.21.15.131:0/3164467515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb610072a90 0x7fb6100cf460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb6100b39c0 tx=0x7fb5f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcec43fc8318ab77 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:37.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.353+0000 7fb60de00640 1 -- 172.21.15.131:0/3164467515 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8037070 con 0x7fb610072a90 2024-04-28T12:38:37.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.353+0000 7fb60de00640 1 -- 172.21.15.131:0/3164467515 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f8032440 con 0x7fb610072a90 2024-04-28T12:38:37.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.353+0000 7fb60de00640 1 -- 172.21.15.131:0/3164467515 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8032810 con 0x7fb610072a90 2024-04-28T12:38:37.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.354+0000 7fb615400640 1 -- 172.21.15.131:0/3164467515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb610072a90 msgr2=0x7fb6100cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.354+0000 7fb615400640 1 --2- 172.21.15.131:0/3164467515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb610072a90 0x7fb6100cf460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb6100b39c0 tx=0x7fb5f802a230 comp rx=0 tx=0).stop 2024-04-28T12:38:37.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.354+0000 7fb615400640 1 -- 172.21.15.131:0/3164467515 shutdown_connections 2024-04-28T12:38:37.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.354+0000 7fb615400640 1 --2- 172.21.15.131:0/3164467515 >> v2:172.21.15.157:3300/0 conn(0x7fb61015d5e0 0x7fb6100cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.354+0000 7fb615400640 1 --2- 172.21.15.131:0/3164467515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb610072a90 0x7fb6100cf460 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.354+0000 7fb615400640 1 -- 172.21.15.131:0/3164467515 >> 172.21.15.131:0/3164467515 conn(0x7fb6100b2900 msgr2=0x7fb6100b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:37.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.354+0000 7fb615400640 1 -- 172.21.15.131:0/3164467515 shutdown_connections 2024-04-28T12:38:37.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.354+0000 7fb615400640 1 -- 172.21.15.131:0/3164467515 wait complete. 2024-04-28T12:38:37.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.356+0000 7fb615400640 1 Processor -- start 2024-04-28T12:38:37.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.356+0000 7fb615400640 1 -- start start 2024-04-28T12:38:37.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb615400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb610072a90 0x7fb610157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:37.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb615400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb610145d80 0x7fb61013f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:37.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb615400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61015d5e0 0x7fb61013d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:37.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb615400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb6100d09f0 con 0x7fb610072a90 2024-04-28T12:38:37.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb615400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb61006e530 con 0x7fb61015d5e0 2024-04-28T12:38:37.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb615400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb6101468c0 con 0x7fb610145d80 2024-04-28T12:38:37.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb60e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb610072a90 0x7fb610157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:37.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb60e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb610072a90 0x7fb610157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37632/0 (socket says 172.21.15.131:37632) 2024-04-28T12:38:37.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb60e800640 1 -- 172.21.15.131:0/2400310135 learned_addr learned my addr 172.21.15.131:0/2400310135 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:37.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb607e00640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb610145d80 0x7fb61013f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:37.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb60f200640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61015d5e0 0x7fb61013d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:37.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb60e800640 1 -- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb610145d80 msgr2=0x7fb61013f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb60e800640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb610145d80 0x7fb61013f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb60e800640 1 -- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61015d5e0 msgr2=0x7fb61013d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb60e800640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61015d5e0 0x7fb61013d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.357+0000 7fb60e800640 1 -- 172.21.15.131:0/2400310135 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6101f2a00 con 0x7fb610072a90 2024-04-28T12:38:37.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.358+0000 7fb60e800640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb610072a90 0x7fb610157900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb60000c460 tx=0x7fb60000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:37.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.358+0000 7fb606a00640 1 -- 172.21.15.131:0/2400310135 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600016020 con 0x7fb610072a90 2024-04-28T12:38:37.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.358+0000 7fb606a00640 1 -- 172.21.15.131:0/2400310135 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb600012870 con 0x7fb610072a90 2024-04-28T12:38:37.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.358+0000 7fb606a00640 1 -- 172.21.15.131:0/2400310135 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb600012c60 con 0x7fb610072a90 2024-04-28T12:38:37.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.358+0000 7fb615400640 1 -- 172.21.15.131:0/2400310135 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61013df80 con 0x7fb610072a90 2024-04-28T12:38:37.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.358+0000 7fb615400640 1 -- 172.21.15.131:0/2400310135 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61013e540 con 0x7fb610072a90 2024-04-28T12:38:37.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.360+0000 7fb615400640 1 -- 172.21.15.131:0/2400310135 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb610069350 con 0x7fb610072a90 2024-04-28T12:38:37.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.360+0000 7fb606a00640 1 -- 172.21.15.131:0/2400310135 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb600019020 con 0x7fb610072a90 2024-04-28T12:38:37.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.361+0000 7fb606a00640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb5ec06c8f0 0x7fb5ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:37.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.361+0000 7fb606a00640 1 -- 172.21.15.131:0/2400310135 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb60008bca0 con 0x7fb610072a90 2024-04-28T12:38:37.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.362+0000 7fb607e00640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb5ec06c8f0 0x7fb5ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:37.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.363+0000 7fb607e00640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb5ec06c8f0 0x7fb5ec06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8034000 tx=0x7fb5f802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:37.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.366+0000 7fb606a00640 1 -- 172.21.15.131:0/2400310135 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb600020030 con 0x7fb610072a90 2024-04-28T12:38:37.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.550+0000 7fb615400640 1 -- 172.21.15.131:0/2400310135 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb61006fb80 con 0x7fb610072a90 2024-04-28T12:38:37.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.551+0000 7fb606a00640 1 -- 172.21.15.131:0/2400310135 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fb600059cc0 con 0x7fb610072a90 2024-04-28T12:38:37.548 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:37.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.553+0000 7fb615400640 1 -- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb5ec06c8f0 msgr2=0x7fb5ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.553+0000 7fb615400640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb5ec06c8f0 0x7fb5ec06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8034000 tx=0x7fb5f802e000 comp rx=0 tx=0).stop 2024-04-28T12:38:37.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.553+0000 7fb615400640 1 -- 172.21.15.131:0/2400310135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb610072a90 msgr2=0x7fb610157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.553+0000 7fb615400640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb610072a90 0x7fb610157900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb60000c460 tx=0x7fb60000c920 comp rx=0 tx=0).stop 2024-04-28T12:38:37.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.553+0000 7fb615400640 1 -- 172.21.15.131:0/2400310135 shutdown_connections 2024-04-28T12:38:37.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.553+0000 7fb615400640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb5ec06c8f0 0x7fb5ec06edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.553+0000 7fb615400640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb61015d5e0 0x7fb61013d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.553+0000 7fb615400640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb610145d80 0x7fb61013f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.553+0000 7fb615400640 1 --2- 172.21.15.131:0/2400310135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb610072a90 0x7fb610157900 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.553+0000 7fb615400640 1 -- 172.21.15.131:0/2400310135 >> 172.21.15.131:0/2400310135 conn(0x7fb6100b2900 msgr2=0x7fb610072380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:37.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.554+0000 7fb615400640 1 -- 172.21.15.131:0/2400310135 shutdown_connections 2024-04-28T12:38:37.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.554+0000 7fb615400640 1 -- 172.21.15.131:0/2400310135 wait complete. 2024-04-28T12:38:37.569 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:37.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.773+0000 7f47a7200640 1 Processor -- start 2024-04-28T12:38:37.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.773+0000 7f47a7200640 1 -- start start 2024-04-28T12:38:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.774+0000 7f47a7200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f47a00b5250 0x7f47a00cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:37.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.774+0000 7f47a7200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47a015d5f0 0x7f47a015da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:37.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.775+0000 7f47a7200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47a006eaf0 con 0x7f47a00cf440 2024-04-28T12:38:37.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.775+0000 7f47a7200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f47a006f420 con 0x7f47a015d5f0 2024-04-28T12:38:37.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.775+0000 7f47a7200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f47a006edd0 con 0x7f47a00b5250 2024-04-28T12:38:37.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.775+0000 7f47a5200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47a015d5f0 0x7f47a015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:37.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.775+0000 7f47a5200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47a015d5f0 0x7f47a015da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37634/0 (socket says 172.21.15.131:37634) 2024-04-28T12:38:37.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.775+0000 7f47a5200640 1 -- 172.21.15.131:0/3933357706 learned_addr learned my addr 172.21.15.131:0/3933357706 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:37.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.775+0000 7f4797400640 1 --2- 172.21.15.131:0/3933357706 >> v2:172.21.15.157:3300/0 conn(0x7f47a00b5250 0x7f47a00cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:37.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.775+0000 7f47a5200640 1 -- 172.21.15.131:0/3933357706 >> v2:172.21.15.157:3300/0 conn(0x7f47a00b5250 msgr2=0x7f47a00cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:37.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.775+0000 7f47a5200640 1 --2- 172.21.15.131:0/3933357706 >> v2:172.21.15.157:3300/0 conn(0x7f47a00b5250 0x7f47a00cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.775+0000 7f47a5200640 1 -- 172.21.15.131:0/3933357706 >> v1:172.21.15.131:6789/0 conn(0x7f47a00cf440 legacy=0x7f47a00cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.775+0000 7f47a5200640 1 -- 172.21.15.131:0/3933357706 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a006f0b0 con 0x7f47a015d5f0 2024-04-28T12:38:37.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.776+0000 7f47a5200640 1 --2- 172.21.15.131:0/3933357706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a015d5f0 0x7f47a015da80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f47a00b59b0 tx=0x7f479802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55a911a44714860 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:37.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.777+0000 7f479f400640 1 -- 172.21.15.131:0/3933357706 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798037070 con 0x7f47a015d5f0 2024-04-28T12:38:37.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.777+0000 7f479f400640 1 -- 172.21.15.131:0/3933357706 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4798032440 con 0x7f47a015d5f0 2024-04-28T12:38:37.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.777+0000 7f479f400640 1 -- 172.21.15.131:0/3933357706 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798032810 con 0x7f47a015d5f0 2024-04-28T12:38:37.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.777+0000 7f47a7200640 1 -- 172.21.15.131:0/3933357706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a015d5f0 msgr2=0x7f47a015da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.777+0000 7f47a7200640 1 --2- 172.21.15.131:0/3933357706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a015d5f0 0x7f47a015da80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f47a00b59b0 tx=0x7f479802a200 comp rx=0 tx=0).stop 2024-04-28T12:38:37.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.777+0000 7f47a7200640 1 -- 172.21.15.131:0/3933357706 shutdown_connections 2024-04-28T12:38:37.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.777+0000 7f47a7200640 1 --2- 172.21.15.131:0/3933357706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a015d5f0 0x7f47a015da80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.777+0000 7f47a7200640 1 --2- 172.21.15.131:0/3933357706 >> v2:172.21.15.157:3300/0 conn(0x7f47a00b5250 0x7f47a00cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.777+0000 7f47a7200640 1 -- 172.21.15.131:0/3933357706 >> 172.21.15.131:0/3933357706 conn(0x7f47a00b2900 msgr2=0x7f47a00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.777+0000 7f47a7200640 1 -- 172.21.15.131:0/3933357706 shutdown_connections 2024-04-28T12:38:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.777+0000 7f47a7200640 1 -- 172.21.15.131:0/3933357706 wait complete. 2024-04-28T12:38:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.779+0000 7f47a7200640 1 Processor -- start 2024-04-28T12:38:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.779+0000 7f47a7200640 1 -- start start 2024-04-28T12:38:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.780+0000 7f47a7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a00b5250 0x7f47a013f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.780+0000 7f47a7200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a00cf440 0x7f47a01f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.780+0000 7f47a7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a015d5f0 0x7f47a01f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.780+0000 7f47a7200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f47a00d0470 con 0x7f47a00cf440 2024-04-28T12:38:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.780+0000 7f47a7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47a006f8b0 con 0x7f47a015d5f0 2024-04-28T12:38:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.780+0000 7f47a7200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f47a015dfc0 con 0x7f47a00b5250 2024-04-28T12:38:37.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.780+0000 7f4797400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a00cf440 0x7f47a01f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.781+0000 7f4797400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a00cf440 0x7f47a01f8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37648/0 (socket says 172.21.15.131:37648) 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.781+0000 7f4797400640 1 -- 172.21.15.131:0/3839655119 learned_addr learned my addr 172.21.15.131:0/3839655119 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.781+0000 7f4797400640 1 -- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a00b5250 msgr2=0x7f47a013f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.781+0000 7f4797400640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a00b5250 0x7f47a013f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.781+0000 7f4797400640 1 -- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a015d5f0 msgr2=0x7f47a01f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.781+0000 7f47a5200640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a015d5f0 0x7f47a01f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.781+0000 7f4797400640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a015d5f0 0x7f47a01f7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.781+0000 7f4797400640 1 -- 172.21.15.131:0/3839655119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a01f93a0 con 0x7f47a00cf440 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.781+0000 7f4797400640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a00cf440 0x7f47a01f8890 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f478800d800 tx=0x7f478800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.781+0000 7f479cc00640 1 -- 172.21.15.131:0/3839655119 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478800b6c0 con 0x7f47a00cf440 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.781+0000 7f479cc00640 1 -- 172.21.15.131:0/3839655119 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4788019070 con 0x7f47a00cf440 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.781+0000 7f479cc00640 1 -- 172.21.15.131:0/3839655119 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478800ba80 con 0x7f47a00cf440 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.782+0000 7f47a7200640 1 -- 172.21.15.131:0/3839655119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a0158fb0 con 0x7f47a00cf440 2024-04-28T12:38:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.782+0000 7f47a7200640 1 -- 172.21.15.131:0/3839655119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a013f670 con 0x7f47a00cf440 2024-04-28T12:38:37.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.783+0000 7f47a7200640 1 -- 172.21.15.131:0/3839655119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a0069350 con 0x7f47a00cf440 2024-04-28T12:38:37.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.784+0000 7f479cc00640 1 -- 172.21.15.131:0/3839655119 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f478800bc20 con 0x7f47a00cf440 2024-04-28T12:38:37.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.784+0000 7f479cc00640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f477c06c8f0 0x7f477c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:37.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.784+0000 7f479cc00640 1 -- 172.21.15.131:0/3839655119 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f478808c400 con 0x7f47a00cf440 2024-04-28T12:38:37.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.785+0000 7f479fe00640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f477c06c8f0 0x7f477c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:37.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.786+0000 7f479fe00640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f477c06c8f0 0x7f477c06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f47900012e0 tx=0x7f4790007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:37.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.789+0000 7f479cc00640 1 -- 172.21.15.131:0/3839655119 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f478805a420 con 0x7f47a00cf440 2024-04-28T12:38:37.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.987+0000 7f47a7200640 1 -- 172.21.15.131:0/3839655119 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f47a0144750 con 0x7f47a00cf440 2024-04-28T12:38:37.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.988+0000 7f479cc00640 1 -- 172.21.15.131:0/3839655119 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f4788059d20 con 0x7f47a00cf440 2024-04-28T12:38:37.984 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:37.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.990+0000 7f47a7200640 1 -- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f477c06c8f0 msgr2=0x7f477c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.990+0000 7f47a7200640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f477c06c8f0 0x7f477c06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f47900012e0 tx=0x7f4790007d50 comp rx=0 tx=0).stop 2024-04-28T12:38:37.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.990+0000 7f47a7200640 1 -- 172.21.15.131:0/3839655119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a00cf440 msgr2=0x7f47a01f8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:37.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.990+0000 7f47a7200640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a00cf440 0x7f47a01f8890 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f478800d800 tx=0x7f478800dcc0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.991+0000 7f47a7200640 1 -- 172.21.15.131:0/3839655119 shutdown_connections 2024-04-28T12:38:37.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.991+0000 7f47a7200640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f477c06c8f0 0x7f477c06edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.991+0000 7f47a7200640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a015d5f0 0x7f47a01f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.991+0000 7f47a7200640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47a00cf440 0x7f47a01f8890 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.991+0000 7f47a7200640 1 --2- 172.21.15.131:0/3839655119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a00b5250 0x7f47a013f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:37.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.991+0000 7f47a7200640 1 -- 172.21.15.131:0/3839655119 >> 172.21.15.131:0/3839655119 conn(0x7f47a00b2900 msgr2=0x7f47a006df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:37.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.991+0000 7f47a7200640 1 -- 172.21.15.131:0/3839655119 shutdown_connections 2024-04-28T12:38:37.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:37.991+0000 7f47a7200640 1 -- 172.21.15.131:0/3839655119 wait complete. 2024-04-28T12:38:38.006 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T12:38:38.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.207+0000 7f1805a00640 1 Processor -- start 2024-04-28T12:38:38.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.208+0000 7f1805a00640 1 -- start start 2024-04-28T12:38:38.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.209+0000 7f1805a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18000b5b80 0x7f18000d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:38.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.209+0000 7f1805a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18000d2800 0x7f18000d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:38.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.209+0000 7f1805a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18000d0480 con 0x7f18000d1620 2024-04-28T12:38:38.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.209+0000 7f1805a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f18000d0ce0 con 0x7f18000d2800 2024-04-28T12:38:38.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.209+0000 7f1805a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f18000d0720 con 0x7f18000b5b80 2024-04-28T12:38:38.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.210+0000 7f1805000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18000d2800 0x7f18000d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:38.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.210+0000 7f17ff400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18000b5b80 0x7f18000d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:38.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.210+0000 7f1805000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f18000d2800 0x7f18000d2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:33016/0 (socket says 172.21.15.131:33016) 2024-04-28T12:38:38.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.210+0000 7f17ff400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18000b5b80 0x7f18000d5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:49464/0 (socket says 172.21.15.131:49464) 2024-04-28T12:38:38.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.210+0000 7f1805000640 1 -- 172.21.15.131:0/3172292011 learned_addr learned my addr 172.21.15.131:0/3172292011 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:38.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.210+0000 7f17ffe00640 1 --1- 172.21.15.131:0/3172292011 >> v1:172.21.15.157:6789/0 conn(0x7f18000d1620 0x7f18000d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.131:51114/0 (socket says 172.21.15.131:51114) 2024-04-28T12:38:38.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.211+0000 7f1805000640 1 -- 172.21.15.131:0/3172292011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18000b5b80 msgr2=0x7f18000d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:38.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.211+0000 7f1805000640 1 --2- 172.21.15.131:0/3172292011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18000b5b80 0x7f18000d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.211+0000 7f1805000640 1 -- 172.21.15.131:0/3172292011 >> v1:172.21.15.157:6789/0 conn(0x7f18000d1620 legacy=0x7f18000d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:38.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.211+0000 7f1805000640 1 -- 172.21.15.131:0/3172292011 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18000d0a00 con 0x7f18000d2800 2024-04-28T12:38:38.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.211+0000 7f17ff400640 1 --2- 172.21.15.131:0/3172292011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18000b5b80 0x7f18000d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:38:38.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.212+0000 7f1805000640 1 --2- 172.21.15.131:0/3172292011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18000d2800 0x7f18000d2c90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009870 tx=0x7f17f802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c216dbb64d0cfc2f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:38.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.212+0000 7f17fea00640 1 -- 172.21.15.131:0/3172292011 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8037070 con 0x7f18000d2800 2024-04-28T12:38:38.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.212+0000 7f17fea00640 1 -- 172.21.15.131:0/3172292011 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f8032440 con 0x7f18000d2800 2024-04-28T12:38:38.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.213+0000 7f17fea00640 1 -- 172.21.15.131:0/3172292011 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8032810 con 0x7f18000d2800 2024-04-28T12:38:38.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.213+0000 7f1805a00640 1 -- 172.21.15.131:0/3172292011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18000d2800 msgr2=0x7f18000d2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:38.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.213+0000 7f1805a00640 1 --2- 172.21.15.131:0/3172292011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18000d2800 0x7f18000d2c90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009870 tx=0x7f17f802a1c0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.213+0000 7f1805a00640 1 -- 172.21.15.131:0/3172292011 shutdown_connections 2024-04-28T12:38:38.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.213+0000 7f1805a00640 1 --2- 172.21.15.131:0/3172292011 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18000d2800 0x7f18000d2c90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.213+0000 7f1805a00640 1 --2- 172.21.15.131:0/3172292011 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18000b5b80 0x7f18000d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.213+0000 7f1805a00640 1 -- 172.21.15.131:0/3172292011 >> 172.21.15.131:0/3172292011 conn(0x7f18000d4c30 msgr2=0x7f18000d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:38.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.213+0000 7f1805a00640 1 -- 172.21.15.131:0/3172292011 shutdown_connections 2024-04-28T12:38:38.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.213+0000 7f1805a00640 1 -- 172.21.15.131:0/3172292011 wait complete. 2024-04-28T12:38:38.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.215+0000 7f1805a00640 1 Processor -- start 2024-04-28T12:38:38.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.215+0000 7f1805a00640 1 -- start start 2024-04-28T12:38:38.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.216+0000 7f1805a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18000b5b80 0x7f18001f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:38.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.216+0000 7f1805a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18000d1620 0x7f18001f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.216+0000 7f1805a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18000d2800 0x7f18001500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.216+0000 7f1805a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f18000d3350 con 0x7f18000d2800 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.216+0000 7f1805a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f18000d31d0 con 0x7f18000d1620 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.216+0000 7f1805a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f18000d34d0 con 0x7f18000b5b80 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.216+0000 7f1805000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18000d2800 0x7f18001500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.216+0000 7f1805000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18000d2800 0x7f18001500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37654/0 (socket says 172.21.15.131:37654) 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.216+0000 7f1805000640 1 -- 172.21.15.131:0/3199558192 learned_addr learned my addr 172.21.15.131:0/3199558192 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.216+0000 7f17ffe00640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18000b5b80 0x7f18001f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.217+0000 7f17ff400640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18000d1620 0x7f18001f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.217+0000 7f1805000640 1 -- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18000b5b80 msgr2=0x7f18001f91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.217+0000 7f1805000640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18000b5b80 0x7f18001f91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.217+0000 7f1805000640 1 -- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18000d1620 msgr2=0x7f18001f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.217+0000 7f1805000640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18000d1620 0x7f18001f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.217+0000 7f1805000640 1 -- 172.21.15.131:0/3199558192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180014f920 con 0x7f18000d2800 2024-04-28T12:38:38.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.217+0000 7f1805000640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18000d2800 0x7f18001500d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f17f80090b0 tx=0x7f17f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:38.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.217+0000 7f17f7e00640 1 -- 172.21.15.131:0/3199558192 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8037040 con 0x7f18000d2800 2024-04-28T12:38:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.217+0000 7f17f7e00640 1 -- 172.21.15.131:0/3199558192 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f8042070 con 0x7f18000d2800 2024-04-28T12:38:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.217+0000 7f17f7e00640 1 -- 172.21.15.131:0/3199558192 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f80308a0 con 0x7f18000d2800 2024-04-28T12:38:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.218+0000 7f1805a00640 1 -- 172.21.15.131:0/3199558192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18001f5950 con 0x7f18000d2800 2024-04-28T12:38:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.218+0000 7f1805a00640 1 -- 172.21.15.131:0/3199558192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18001f5f10 con 0x7f18000d2800 2024-04-28T12:38:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.219+0000 7f1805a00640 1 -- 172.21.15.131:0/3199558192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1800069350 con 0x7f18000d2800 2024-04-28T12:38:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.220+0000 7f17f7e00640 1 -- 172.21.15.131:0/3199558192 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17f802ad10 con 0x7f18000d2800 2024-04-28T12:38:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.221+0000 7f17f7e00640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f17dc06c8f0 0x7f17dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.221+0000 7f17f7e00640 1 -- 172.21.15.131:0/3199558192 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f17f80ac5a0 con 0x7f18000d2800 2024-04-28T12:38:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.222+0000 7f17ffe00640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f17dc06c8f0 0x7f17dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:38.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.223+0000 7f17ffe00640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f17dc06c8f0 0x7f17dc06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f18000b3f40 tx=0x7f17f0007cc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.225+0000 7f17f7e00640 1 -- 172.21.15.131:0/3199558192 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17f8079ee0 con 0x7f18000d2800 2024-04-28T12:38:38.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.440+0000 7f1805a00640 1 -- 172.21.15.131:0/3199558192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f18000b8a70 con 0x7f18000d2800 2024-04-28T12:38:38.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.441+0000 7f17f7e00640 1 -- 172.21.15.131:0/3199558192 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f17f807e1e0 con 0x7f18000d2800 2024-04-28T12:38:38.438 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-04-28T12:38:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.444+0000 7f1805a00640 1 -- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f17dc06c8f0 msgr2=0x7f17dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.444+0000 7f1805a00640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f17dc06c8f0 0x7f17dc06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f18000b3f40 tx=0x7f17f0007cc0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.444+0000 7f1805a00640 1 -- 172.21.15.131:0/3199558192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18000d2800 msgr2=0x7f18001500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.444+0000 7f1805a00640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18000d2800 0x7f18001500d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f17f80090b0 tx=0x7f17f80046c0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.444+0000 7f1805a00640 1 -- 172.21.15.131:0/3199558192 shutdown_connections 2024-04-28T12:38:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.444+0000 7f1805a00640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f17dc06c8f0 0x7f17dc06edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.444+0000 7f1805a00640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18000d2800 0x7f18001500d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.444+0000 7f1805a00640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18000d1620 0x7f18001f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.444+0000 7f1805a00640 1 --2- 172.21.15.131:0/3199558192 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18000b5b80 0x7f18001f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.444+0000 7f1805a00640 1 -- 172.21.15.131:0/3199558192 >> 172.21.15.131:0/3199558192 conn(0x7f18000d4c30 msgr2=0x7f18000d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.444+0000 7f1805a00640 1 -- 172.21.15.131:0/3199558192 shutdown_connections 2024-04-28T12:38:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.444+0000 7f1805a00640 1 -- 172.21.15.131:0/3199558192 wait complete. 2024-04-28T12:38:38.460 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:38:38.460 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:38:38.517 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:38.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.712+0000 7f4574400640 1 Processor -- start 2024-04-28T12:38:38.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.713+0000 7f4574400640 1 -- start start 2024-04-28T12:38:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.714+0000 7f4574400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f456c15cab0 0x7f456c0afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.714+0000 7f4574400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f456c0d1250 0x7f456c15f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.714+0000 7f4574400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f456c0b0650 con 0x7f456c0b0020 2024-04-28T12:38:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.714+0000 7f4574400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f456c0d0900 con 0x7f456c15cab0 2024-04-28T12:38:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.714+0000 7f4574400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f456c1635a0 con 0x7f456c0d1250 2024-04-28T12:38:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.715+0000 7f4571a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f456c15cab0 0x7f456c0afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:38.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.715+0000 7f4571a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f456c15cab0 0x7f456c0afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37662/0 (socket says 172.21.15.131:37662) 2024-04-28T12:38:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.715+0000 7f4571a00640 1 -- 172.21.15.131:0/3978507566 learned_addr learned my addr 172.21.15.131:0/3978507566 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.715+0000 7f4572400640 1 --2- 172.21.15.131:0/3978507566 >> v2:172.21.15.157:3300/0 conn(0x7f456c0d1250 0x7f456c15f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:38.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.715+0000 7f4571a00640 1 -- 172.21.15.131:0/3978507566 >> v2:172.21.15.157:3300/0 conn(0x7f456c0d1250 msgr2=0x7f456c15f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.715+0000 7f4571a00640 1 --2- 172.21.15.131:0/3978507566 >> v2:172.21.15.157:3300/0 conn(0x7f456c0d1250 0x7f456c15f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.715+0000 7f4571a00640 1 -- 172.21.15.131:0/3978507566 >> v1:172.21.15.157:6789/0 conn(0x7f456c0b0020 legacy=0x7f456c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.715+0000 7f4571a00640 1 -- 172.21.15.131:0/3978507566 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c163a30 con 0x7f456c15cab0 2024-04-28T12:38:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.715+0000 7f4572400640 1 --2- 172.21.15.131:0/3978507566 >> v2:172.21.15.157:3300/0 conn(0x7f456c0d1250 0x7f456c15f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:38.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.717+0000 7f4571a00640 1 --2- 172.21.15.131:0/3978507566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f456c15cab0 0x7f456c0afae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f456c0b3920 tx=0x7f456002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc7b45d4308ca5c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.717+0000 7f456be00640 1 -- 172.21.15.131:0/3978507566 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560037070 con 0x7f456c15cab0 2024-04-28T12:38:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.717+0000 7f456be00640 1 -- 172.21.15.131:0/3978507566 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4560032440 con 0x7f456c15cab0 2024-04-28T12:38:38.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.717+0000 7f456be00640 1 -- 172.21.15.131:0/3978507566 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560032810 con 0x7f456c15cab0 2024-04-28T12:38:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.718+0000 7f4574400640 1 -- 172.21.15.131:0/3978507566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f456c15cab0 msgr2=0x7f456c0afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.718+0000 7f4574400640 1 --2- 172.21.15.131:0/3978507566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f456c15cab0 0x7f456c0afae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f456c0b3920 tx=0x7f456002a230 comp rx=0 tx=0).stop 2024-04-28T12:38:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.718+0000 7f4574400640 1 -- 172.21.15.131:0/3978507566 shutdown_connections 2024-04-28T12:38:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.718+0000 7f4574400640 1 --2- 172.21.15.131:0/3978507566 >> v2:172.21.15.157:3300/0 conn(0x7f456c0d1250 0x7f456c15f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.718+0000 7f4574400640 1 --2- 172.21.15.131:0/3978507566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f456c15cab0 0x7f456c0afae0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.718+0000 7f4574400640 1 -- 172.21.15.131:0/3978507566 >> 172.21.15.131:0/3978507566 conn(0x7f456c0b2900 msgr2=0x7f456c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.718+0000 7f4574400640 1 -- 172.21.15.131:0/3978507566 shutdown_connections 2024-04-28T12:38:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.718+0000 7f4574400640 1 -- 172.21.15.131:0/3978507566 wait complete. 2024-04-28T12:38:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.720+0000 7f4574400640 1 Processor -- start 2024-04-28T12:38:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.720+0000 7f4574400640 1 -- start start 2024-04-28T12:38:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4574400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c0b0020 0x7f456c1e89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4574400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c0d1250 0x7f456c1f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4574400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f456c15cab0 0x7f456c160e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4574400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f456c163230 con 0x7f456c15cab0 2024-04-28T12:38:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4574400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f456c162f50 con 0x7f456c0d1250 2024-04-28T12:38:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4574400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f456c15ce90 con 0x7f456c0b0020 2024-04-28T12:38:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4572400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f456c15cab0 0x7f456c160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4572400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f456c15cab0 0x7f456c160e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37676/0 (socket says 172.21.15.131:37676) 2024-04-28T12:38:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4572400640 1 -- 172.21.15.131:0/2567410237 learned_addr learned my addr 172.21.15.131:0/2567410237 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4571a00640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c0b0020 0x7f456c1e89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4571000640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c0d1250 0x7f456c1f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4572400640 1 -- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c0b0020 msgr2=0x7f456c1e89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4572400640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c0b0020 0x7f456c1e89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.721+0000 7f4572400640 1 -- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c0d1250 msgr2=0x7f456c1f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.722+0000 7f4572400640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c0d1250 0x7f456c1f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.722+0000 7f4572400640 1 -- 172.21.15.131:0/2567410237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c1e90b0 con 0x7f456c15cab0 2024-04-28T12:38:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.722+0000 7f4572400640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f456c15cab0 0x7f456c160e40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f456400cef0 tx=0x7f456400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.722+0000 7f4569600640 1 -- 172.21.15.131:0/2567410237 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456401a070 con 0x7f456c15cab0 2024-04-28T12:38:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.722+0000 7f4569600640 1 -- 172.21.15.131:0/2567410237 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456400d860 con 0x7f456c15cab0 2024-04-28T12:38:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.722+0000 7f4569600640 1 -- 172.21.15.131:0/2567410237 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456400dc50 con 0x7f456c15cab0 2024-04-28T12:38:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.722+0000 7f4574400640 1 -- 172.21.15.131:0/2567410237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c1e9390 con 0x7f456c15cab0 2024-04-28T12:38:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.722+0000 7f4574400640 1 -- 172.21.15.131:0/2567410237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c1e9950 con 0x7f456c15cab0 2024-04-28T12:38:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.724+0000 7f4574400640 1 -- 172.21.15.131:0/2567410237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456c069350 con 0x7f456c15cab0 2024-04-28T12:38:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.725+0000 7f4569600640 1 -- 172.21.15.131:0/2567410237 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f456401d070 con 0x7f456c15cab0 2024-04-28T12:38:38.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.725+0000 7f4569600640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f454c06c8f0 0x7f454c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:38.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.726+0000 7f4569600640 1 -- 172.21.15.131:0/2567410237 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f456408c8c0 con 0x7f456c15cab0 2024-04-28T12:38:38.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.726+0000 7f4571a00640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f454c06c8f0 0x7f454c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:38.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.727+0000 7f4571a00640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f454c06c8f0 0x7f454c06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4560034000 tx=0x7f456002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:38.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.730+0000 7f4569600640 1 -- 172.21.15.131:0/2567410237 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f456405a910 con 0x7f456c15cab0 2024-04-28T12:38:38.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.937+0000 7f4574400640 1 -- 172.21.15.131:0/2567410237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f456c161380 con 0x7f456c15cab0 2024-04-28T12:38:38.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.939+0000 7f4569600640 1 -- 172.21.15.131:0/2567410237 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7f456405a210 con 0x7f456c15cab0 2024-04-28T12:38:38.936 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:38.937 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:38.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.942+0000 7f4574400640 1 -- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f454c06c8f0 msgr2=0x7f454c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.942+0000 7f4574400640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f454c06c8f0 0x7f454c06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4560034000 tx=0x7f456002e000 comp rx=0 tx=0).stop 2024-04-28T12:38:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.942+0000 7f4574400640 1 -- 172.21.15.131:0/2567410237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f456c15cab0 msgr2=0x7f456c160e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.942+0000 7f4574400640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f456c15cab0 0x7f456c160e40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f456400cef0 tx=0x7f456400a710 comp rx=0 tx=0).stop 2024-04-28T12:38:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.943+0000 7f4574400640 1 -- 172.21.15.131:0/2567410237 shutdown_connections 2024-04-28T12:38:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.943+0000 7f4574400640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f454c06c8f0 0x7f454c06edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.943+0000 7f4574400640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f456c15cab0 0x7f456c160e40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.943+0000 7f4574400640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f456c0d1250 0x7f456c1f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.943+0000 7f4574400640 1 --2- 172.21.15.131:0/2567410237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c0b0020 0x7f456c1e89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.943+0000 7f4574400640 1 -- 172.21.15.131:0/2567410237 >> 172.21.15.131:0/2567410237 conn(0x7f456c0b2900 msgr2=0x7f456c0b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.943+0000 7f4574400640 1 -- 172.21.15.131:0/2567410237 shutdown_connections 2024-04-28T12:38:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:38.943+0000 7f4574400640 1 -- 172.21.15.131:0/2567410237 wait complete. 2024-04-28T12:38:38.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:38.959 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:39.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.157+0000 7f13a2a00640 1 Processor -- start 2024-04-28T12:38:39.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.158+0000 7f13a2a00640 1 -- start start 2024-04-28T12:38:39.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.159+0000 7f13a2a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f139c0d3c10 0x7f139c0b9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:39.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.159+0000 7f13a2a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f139c0b8cd0 con 0x7f139c0b87c0 2024-04-28T12:38:39.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.159+0000 7f13a2a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f139c0b8fb0 con 0x7f139c15f350 2024-04-28T12:38:39.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.159+0000 7f13a2a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f139c0b9290 con 0x7f139c0d3c10 2024-04-28T12:38:39.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.159+0000 7f139b400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f139c0d3c10 0x7f139c0b9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:39.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.159+0000 7f139b400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f139c0d3c10 0x7f139c0b9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37692/0 (socket says 172.21.15.131:37692) 2024-04-28T12:38:39.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.159+0000 7f139b400640 1 -- 172.21.15.131:0/3859403263 learned_addr learned my addr 172.21.15.131:0/3859403263 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:39.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.159+0000 7f13a0a00640 1 --1- 172.21.15.131:0/3859403263 >> v1:172.21.15.131:6789/0 conn(0x7f139c0b87c0 0x7f139c0ba2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47136/0 (socket says 172.21.15.131:47136) 2024-04-28T12:38:39.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.160+0000 7f139b400640 1 -- 172.21.15.131:0/3859403263 >> v1:172.21.15.157:6789/0 conn(0x7f139c15f350 legacy=0x7f139c0d3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:39.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.160+0000 7f139b400640 1 -- 172.21.15.131:0/3859403263 >> v1:172.21.15.131:6789/0 conn(0x7f139c0b87c0 legacy=0x7f139c0ba2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.160+0000 7f139b400640 1 -- 172.21.15.131:0/3859403263 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139c0b9570 con 0x7f139c0d3c10 2024-04-28T12:38:39.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.160+0000 7f139aa00640 1 -- 172.21.15.131:0/3859403263 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3440642678 0 0) 0x7f139c0b8cd0 con 0x7f139c0b87c0 2024-04-28T12:38:39.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.161+0000 7f139b400640 1 --2- 172.21.15.131:0/3859403263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f139c0d3c10 0x7f139c0b9d70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1384009870 tx=0x7f138402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ac7672bc781e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:39.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.161+0000 7f139aa00640 1 -- 172.21.15.131:0/3859403263 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384037070 con 0x7f139c0d3c10 2024-04-28T12:38:39.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.161+0000 7f139aa00640 1 -- 172.21.15.131:0/3859403263 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1384032440 con 0x7f139c0d3c10 2024-04-28T12:38:39.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.161+0000 7f139aa00640 1 -- 172.21.15.131:0/3859403263 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384032810 con 0x7f139c0d3c10 2024-04-28T12:38:39.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.162+0000 7f13a2a00640 1 -- 172.21.15.131:0/3859403263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f139c0d3c10 msgr2=0x7f139c0b9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.162+0000 7f13a2a00640 1 --2- 172.21.15.131:0/3859403263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f139c0d3c10 0x7f139c0b9d70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1384009870 tx=0x7f138402a2a0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.162+0000 7f13a2a00640 1 -- 172.21.15.131:0/3859403263 shutdown_connections 2024-04-28T12:38:39.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.162+0000 7f13a2a00640 1 --2- 172.21.15.131:0/3859403263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f139c0d3c10 0x7f139c0b9d70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.162+0000 7f13a2a00640 1 -- 172.21.15.131:0/3859403263 >> 172.21.15.131:0/3859403263 conn(0x7f139c0d4fb0 msgr2=0x7f139c0d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:39.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.162+0000 7f13a2a00640 1 -- 172.21.15.131:0/3859403263 shutdown_connections 2024-04-28T12:38:39.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.162+0000 7f13a2a00640 1 -- 172.21.15.131:0/3859403263 wait complete. 2024-04-28T12:38:39.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.164+0000 7f13a2a00640 1 Processor -- start 2024-04-28T12:38:39.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.164+0000 7f13a2a00640 1 -- start start 2024-04-28T12:38:39.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.165+0000 7f13a2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f139c0b87c0 0x7f139c153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:39.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.165+0000 7f13a2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f139c0d3c10 0x7f139c0a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:39.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.165+0000 7f13a2a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f139c15f350 0x7f139c0cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:39.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.165+0000 7f13a2a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f139c0d0ea0 con 0x7f139c0b87c0 2024-04-28T12:38:39.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.165+0000 7f13a2a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f139c0d0d20 con 0x7f139c0d3c10 2024-04-28T12:38:39.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.165+0000 7f13a2a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f139c15f730 con 0x7f139c15f350 2024-04-28T12:38:39.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.165+0000 7f139be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f139c0b87c0 0x7f139c153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:39.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.165+0000 7f139be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f139c0b87c0 0x7f139c153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37694/0 (socket says 172.21.15.131:37694) 2024-04-28T12:38:39.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.165+0000 7f139be00640 1 -- 172.21.15.131:0/3667942920 learned_addr learned my addr 172.21.15.131:0/3667942920 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f13a0a00640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f139c15f350 0x7f139c0cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f139b400640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f139c0d3c10 0x7f139c0a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f139be00640 1 -- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f139c15f350 msgr2=0x7f139c0cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f139be00640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f139c15f350 0x7f139c0cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f139be00640 1 -- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f139c0d3c10 msgr2=0x7f139c0a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f139be00640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f139c0d3c10 0x7f139c0a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f139be00640 1 -- 172.21.15.131:0/3667942920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139c15ca20 con 0x7f139c0b87c0 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f13a0a00640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f139c15f350 0x7f139c0cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f139b400640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f139c0d3c10 0x7f139c0a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f139be00640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f139c0b87c0 0x7f139c153530 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f138c00ba80 tx=0x7f138c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f1393e00640 1 -- 172.21.15.131:0/3667942920 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c00cc00 con 0x7f139c0b87c0 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f1393e00640 1 -- 172.21.15.131:0/3667942920 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f138c00cda0 con 0x7f139c0b87c0 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f1393e00640 1 -- 172.21.15.131:0/3667942920 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c003bb0 con 0x7f139c0b87c0 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.166+0000 7f13a2a00640 1 -- 172.21.15.131:0/3667942920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139c0a2730 con 0x7f139c0b87c0 2024-04-28T12:38:39.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.167+0000 7f13a2a00640 1 -- 172.21.15.131:0/3667942920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f139c0a1430 con 0x7f139c0b87c0 2024-04-28T12:38:39.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.168+0000 7f13a2a00640 1 -- 172.21.15.131:0/3667942920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139c069350 con 0x7f139c0b87c0 2024-04-28T12:38:39.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.169+0000 7f1393e00640 1 -- 172.21.15.131:0/3667942920 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f138c014440 con 0x7f139c0b87c0 2024-04-28T12:38:39.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.169+0000 7f1393e00640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f137806c8f0 0x7f137806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:39.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.170+0000 7f1393e00640 1 -- 172.21.15.131:0/3667942920 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f138c08bea0 con 0x7f139c0b87c0 2024-04-28T12:38:39.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.170+0000 7f139b400640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f137806c8f0 0x7f137806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:39.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.171+0000 7f139b400640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f137806c8f0 0x7f137806edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1384034000 tx=0x7f138402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:39.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.174+0000 7f1393e00640 1 -- 172.21.15.131:0/3667942920 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f139c15ca20 con 0x7f139c0b87c0 2024-04-28T12:38:39.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.373+0000 7f13a2a00640 1 -- 172.21.15.131:0/3667942920 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f139c0d39f0 con 0x7f139c0b87c0 2024-04-28T12:38:39.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.376+0000 7f1393e00640 1 -- 172.21.15.131:0/3667942920 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f138c059ed0 con 0x7f139c0b87c0 2024-04-28T12:38:39.373 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.379+0000 7f13a2a00640 1 -- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f137806c8f0 msgr2=0x7f137806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.379+0000 7f13a2a00640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f137806c8f0 0x7f137806edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1384034000 tx=0x7f138402e000 comp rx=0 tx=0).stop 2024-04-28T12:38:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.379+0000 7f13a2a00640 1 -- 172.21.15.131:0/3667942920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f139c0b87c0 msgr2=0x7f139c153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.379+0000 7f13a2a00640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f139c0b87c0 0x7f139c153530 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f138c00ba80 tx=0x7f138c00bf40 comp rx=0 tx=0).stop 2024-04-28T12:38:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.379+0000 7f13a2a00640 1 -- 172.21.15.131:0/3667942920 shutdown_connections 2024-04-28T12:38:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.379+0000 7f13a2a00640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f137806c8f0 0x7f137806edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.379+0000 7f13a2a00640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f139c15f350 0x7f139c0cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.379+0000 7f13a2a00640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f139c0d3c10 0x7f139c0a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.379+0000 7f13a2a00640 1 --2- 172.21.15.131:0/3667942920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f139c0b87c0 0x7f139c153530 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.379+0000 7f13a2a00640 1 -- 172.21.15.131:0/3667942920 >> 172.21.15.131:0/3667942920 conn(0x7f139c0d4fb0 msgr2=0x7f139c0d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.380+0000 7f13a2a00640 1 -- 172.21.15.131:0/3667942920 shutdown_connections 2024-04-28T12:38:39.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.380+0000 7f13a2a00640 1 -- 172.21.15.131:0/3667942920 wait complete. 2024-04-28T12:38:39.396 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:39.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.600+0000 7f27a1800640 1 Processor -- start 2024-04-28T12:38:39.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.600+0000 7f27a1800640 1 -- start start 2024-04-28T12:38:39.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.602+0000 7f27a1800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c0723f0 0x7f279c163b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:39.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.602+0000 7f27a1800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f279c073660 con 0x7f279c06f380 2024-04-28T12:38:39.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.602+0000 7f27a1800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f279c06e190 con 0x7f279c14d200 2024-04-28T12:38:39.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.602+0000 7f27a1800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f279c06df40 con 0x7f279c0723f0 2024-04-28T12:38:39.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.602+0000 7f279b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f279c06f380 0x7f279c0b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47138/0 (socket says 172.21.15.131:47138) 2024-04-28T12:38:39.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.603+0000 7f279b400640 1 -- 172.21.15.131:0/4155673858 learned_addr learned my addr 172.21.15.131:0/4155673858 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:39.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.603+0000 7f2793e00640 1 --2- 172.21.15.131:0/4155673858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c0723f0 0x7f279c163b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:39.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.603+0000 7f279a000640 1 -- 172.21.15.131:0/4155673858 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3624958569 0 0) 0x7f279c073660 con 0x7f279c06f380 2024-04-28T12:38:39.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.603+0000 7f279a000640 1 -- 172.21.15.131:0/4155673858 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2780003520 con 0x7f279c06f380 2024-04-28T12:38:39.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.604+0000 7f279a000640 1 -- 172.21.15.131:0/4155673858 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2999725407 0 0) 0x7f279c06e190 con 0x7f279c14d200 2024-04-28T12:38:39.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.604+0000 7f279a000640 1 -- 172.21.15.131:0/4155673858 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f279c073660 con 0x7f279c14d200 2024-04-28T12:38:39.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.604+0000 7f2793e00640 1 -- 172.21.15.131:0/4155673858 >> v1:172.21.15.157:6789/0 conn(0x7f279c14d200 legacy=0x7f279c06ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.604+0000 7f2793e00640 1 -- 172.21.15.131:0/4155673858 >> v1:172.21.15.131:6789/0 conn(0x7f279c06f380 legacy=0x7f279c0b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.604+0000 7f2793e00640 1 -- 172.21.15.131:0/4155673858 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c06e470 con 0x7f279c0723f0 2024-04-28T12:38:39.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.605+0000 7f2793e00640 1 --2- 172.21.15.131:0/4155673858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c0723f0 0x7f279c163b70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f27840097d0 tx=0x7f278402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3eb9d2e338d52c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:39.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.605+0000 7f279a000640 1 -- 172.21.15.131:0/4155673858 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784037070 con 0x7f279c0723f0 2024-04-28T12:38:39.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.605+0000 7f279a000640 1 -- 172.21.15.131:0/4155673858 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2784032440 con 0x7f279c0723f0 2024-04-28T12:38:39.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.605+0000 7f279a000640 1 -- 172.21.15.131:0/4155673858 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27840327f0 con 0x7f279c0723f0 2024-04-28T12:38:39.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.606+0000 7f27a1800640 1 -- 172.21.15.131:0/4155673858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c0723f0 msgr2=0x7f279c163b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.606+0000 7f27a1800640 1 --2- 172.21.15.131:0/4155673858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c0723f0 0x7f279c163b70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f27840097d0 tx=0x7f278402a210 comp rx=0 tx=0).stop 2024-04-28T12:38:39.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.606+0000 7f27a1800640 1 -- 172.21.15.131:0/4155673858 shutdown_connections 2024-04-28T12:38:39.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.606+0000 7f27a1800640 1 --2- 172.21.15.131:0/4155673858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c0723f0 0x7f279c163b70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.606+0000 7f27a1800640 1 -- 172.21.15.131:0/4155673858 >> 172.21.15.131:0/4155673858 conn(0x7f279c0b2900 msgr2=0x7f279c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:39.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.606+0000 7f27a1800640 1 -- 172.21.15.131:0/4155673858 shutdown_connections 2024-04-28T12:38:39.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.606+0000 7f27a1800640 1 -- 172.21.15.131:0/4155673858 wait complete. 2024-04-28T12:38:39.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.609+0000 7f27a1800640 1 Processor -- start 2024-04-28T12:38:39.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.609+0000 7f27a1800640 1 -- start start 2024-04-28T12:38:39.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.610+0000 7f27a1800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c06f380 0x7f279c15b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:39.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.610+0000 7f27a1800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0723f0 0x7f279c15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:39.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.610+0000 7f27a1800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f279c14d200 0x7f279c14a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.610+0000 7f27a1800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f279c073440 con 0x7f279c14d200 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.610+0000 7f27a1800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f279c0732c0 con 0x7f279c0723f0 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.610+0000 7f27a1800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f279c06f810 con 0x7f279c06f380 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.610+0000 7f279b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f279c14d200 0x7f279c14a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.611+0000 7f279b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f279c14d200 0x7f279c14a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37710/0 (socket says 172.21.15.131:37710) 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.611+0000 7f279b400640 1 -- 172.21.15.131:0/310849970 learned_addr learned my addr 172.21.15.131:0/310849970 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.611+0000 7f279aa00640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c06f380 0x7f279c15b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.611+0000 7f2793e00640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0723f0 0x7f279c15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.611+0000 7f279b400640 1 -- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c06f380 msgr2=0x7f279c15b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.611+0000 7f279b400640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c06f380 0x7f279c15b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.611+0000 7f279b400640 1 -- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0723f0 msgr2=0x7f279c15d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.611+0000 7f279b400640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0723f0 0x7f279c15d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.611+0000 7f279b400640 1 -- 172.21.15.131:0/310849970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c14ac60 con 0x7f279c14d200 2024-04-28T12:38:39.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.611+0000 7f279b400640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f279c14d200 0x7f279c14a0e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f279400caf0 tx=0x7f279400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:39.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.612+0000 7f2792a00640 1 -- 172.21.15.131:0/310849970 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794017070 con 0x7f279c14d200 2024-04-28T12:38:39.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.612+0000 7f2792a00640 1 -- 172.21.15.131:0/310849970 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279400aca0 con 0x7f279c14d200 2024-04-28T12:38:39.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.612+0000 7f2792a00640 1 -- 172.21.15.131:0/310849970 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794012620 con 0x7f279c14d200 2024-04-28T12:38:39.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.612+0000 7f27a1800640 1 -- 172.21.15.131:0/310849970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279c14af40 con 0x7f279c14d200 2024-04-28T12:38:39.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.612+0000 7f27a1800640 1 -- 172.21.15.131:0/310849970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279c14b500 con 0x7f279c14d200 2024-04-28T12:38:39.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.613+0000 7f27a1800640 1 -- 172.21.15.131:0/310849970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f279c069350 con 0x7f279c14d200 2024-04-28T12:38:39.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.615+0000 7f2792a00640 1 -- 172.21.15.131:0/310849970 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f279401a070 con 0x7f279c14d200 2024-04-28T12:38:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.615+0000 7f2792a00640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f277806c8f0 0x7f277806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.616+0000 7f2792a00640 1 -- 172.21.15.131:0/310849970 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f279408bd10 con 0x7f279c14d200 2024-04-28T12:38:39.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.616+0000 7f279aa00640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f277806c8f0 0x7f277806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:39.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.617+0000 7f279aa00640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f277806c8f0 0x7f277806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f278c0011e0 tx=0x7f278c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:39.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.620+0000 7f2792a00640 1 -- 172.21.15.131:0/310849970 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2794091020 con 0x7f279c14d200 2024-04-28T12:38:39.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.821+0000 7f27a1800640 1 -- 172.21.15.131:0/310849970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f279c0d06c0 con 0x7f279c14d200 2024-04-28T12:38:39.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.821+0000 7f2792a00640 1 -- 172.21.15.131:0/310849970 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f2794059d30 con 0x7f279c14d200 2024-04-28T12:38:39.818 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:39.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.824+0000 7f27a1800640 1 -- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f277806c8f0 msgr2=0x7f277806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.824+0000 7f27a1800640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f277806c8f0 0x7f277806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f278c0011e0 tx=0x7f278c009040 comp rx=0 tx=0).stop 2024-04-28T12:38:39.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.824+0000 7f27a1800640 1 -- 172.21.15.131:0/310849970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f279c14d200 msgr2=0x7f279c14a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:39.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.824+0000 7f27a1800640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f279c14d200 0x7f279c14a0e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f279400caf0 tx=0x7f279400a060 comp rx=0 tx=0).stop 2024-04-28T12:38:39.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.824+0000 7f27a1800640 1 -- 172.21.15.131:0/310849970 shutdown_connections 2024-04-28T12:38:39.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.824+0000 7f27a1800640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f277806c8f0 0x7f277806edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.824+0000 7f27a1800640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f279c14d200 0x7f279c14a0e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.824+0000 7f27a1800640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f279c0723f0 0x7f279c15d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.824+0000 7f27a1800640 1 --2- 172.21.15.131:0/310849970 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f279c06f380 0x7f279c15b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:39.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.824+0000 7f27a1800640 1 -- 172.21.15.131:0/310849970 >> 172.21.15.131:0/310849970 conn(0x7f279c0b2900 msgr2=0x7f279c0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:39.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.824+0000 7f27a1800640 1 -- 172.21.15.131:0/310849970 shutdown_connections 2024-04-28T12:38:39.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:39.824+0000 7f27a1800640 1 -- 172.21.15.131:0/310849970 wait complete. 2024-04-28T12:38:39.841 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.053+0000 7f6e4da00640 1 Processor -- start 2024-04-28T12:38:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.053+0000 7f6e4da00640 1 -- start start 2024-04-28T12:38:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.055+0000 7f6e4da00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6e48072a90 0x7f6e480cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.055+0000 7f6e4da00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6e4815d5e0 0x7f6e480cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.055+0000 7f6e4da00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e48146160 con 0x7f6e48145d80 2024-04-28T12:38:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.055+0000 7f6e4da00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6e4806dfb0 con 0x7f6e4815d5e0 2024-04-28T12:38:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.055+0000 7f6e4da00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6e48074180 con 0x7f6e48072a90 2024-04-28T12:38:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.055+0000 7f6e47400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6e4815d5e0 0x7f6e480cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.055+0000 7f6e47400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6e4815d5e0 0x7f6e480cf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37716/0 (socket says 172.21.15.131:37716) 2024-04-28T12:38:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.055+0000 7f6e47400640 1 -- 172.21.15.131:0/2035086834 learned_addr learned my addr 172.21.15.131:0/2035086834 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.055+0000 7f6e46000640 1 --2- 172.21.15.131:0/2035086834 >> v2:172.21.15.157:3300/0 conn(0x7f6e48072a90 0x7f6e480cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.056+0000 7f6e47400640 1 -- 172.21.15.131:0/2035086834 >> v2:172.21.15.157:3300/0 conn(0x7f6e48072a90 msgr2=0x7f6e480cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.056+0000 7f6e47400640 1 --2- 172.21.15.131:0/2035086834 >> v2:172.21.15.157:3300/0 conn(0x7f6e48072a90 0x7f6e480cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.056+0000 7f6e47400640 1 -- 172.21.15.131:0/2035086834 >> v1:172.21.15.157:6789/0 conn(0x7f6e48145d80 legacy=0x7f6e48072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.056+0000 7f6e47400640 1 -- 172.21.15.131:0/2035086834 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e480d0710 con 0x7f6e4815d5e0 2024-04-28T12:38:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.057+0000 7f6e47400640 1 --2- 172.21.15.131:0/2035086834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4815d5e0 0x7f6e480cf9a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6e48070130 tx=0x7f6e4002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e77c1c5f6f8e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.058+0000 7f6e45600640 1 -- 172.21.15.131:0/2035086834 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40036070 con 0x7f6e4815d5e0 2024-04-28T12:38:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.058+0000 7f6e45600640 1 -- 172.21.15.131:0/2035086834 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e40031600 con 0x7f6e4815d5e0 2024-04-28T12:38:40.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.058+0000 7f6e45600640 1 -- 172.21.15.131:0/2035086834 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e400319d0 con 0x7f6e4815d5e0 2024-04-28T12:38:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.058+0000 7f6e4da00640 1 -- 172.21.15.131:0/2035086834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4815d5e0 msgr2=0x7f6e480cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.058+0000 7f6e4da00640 1 --2- 172.21.15.131:0/2035086834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4815d5e0 0x7f6e480cf9a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6e48070130 tx=0x7f6e4002a6f0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.059+0000 7f6e4da00640 1 -- 172.21.15.131:0/2035086834 shutdown_connections 2024-04-28T12:38:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.059+0000 7f6e4da00640 1 --2- 172.21.15.131:0/2035086834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4815d5e0 0x7f6e480cf9a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.059+0000 7f6e4da00640 1 --2- 172.21.15.131:0/2035086834 >> v2:172.21.15.157:3300/0 conn(0x7f6e48072a90 0x7f6e480cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.059+0000 7f6e4da00640 1 -- 172.21.15.131:0/2035086834 >> 172.21.15.131:0/2035086834 conn(0x7f6e480b2900 msgr2=0x7f6e480b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:40.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.059+0000 7f6e4da00640 1 -- 172.21.15.131:0/2035086834 shutdown_connections 2024-04-28T12:38:40.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.059+0000 7f6e4da00640 1 -- 172.21.15.131:0/2035086834 wait complete. 2024-04-28T12:38:40.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.061+0000 7f6e4da00640 1 Processor -- start 2024-04-28T12:38:40.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.061+0000 7f6e4da00640 1 -- start start 2024-04-28T12:38:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.062+0000 7f6e4da00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e48072a90 0x7f6e480b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e4da00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e48145d80 0x7f6e481f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e4da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4815d5e0 0x7f6e481419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e4da00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480d09f0 con 0x7f6e4815d5e0 2024-04-28T12:38:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e4da00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4806e530 con 0x7f6e48072a90 2024-04-28T12:38:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e4da00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6e481468c0 con 0x7f6e48145d80 2024-04-28T12:38:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e47400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4815d5e0 0x7f6e481419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e47400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4815d5e0 0x7f6e481419c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37728/0 (socket says 172.21.15.131:37728) 2024-04-28T12:38:40.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e47400640 1 -- 172.21.15.131:0/689268357 learned_addr learned my addr 172.21.15.131:0/689268357 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e46a00640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e48072a90 0x7f6e480b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e46000640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e48145d80 0x7f6e481f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e47400640 1 -- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e48145d80 msgr2=0x7f6e481f8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e47400640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e48145d80 0x7f6e481f8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e47400640 1 -- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e48072a90 msgr2=0x7f6e480b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e47400640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e48072a90 0x7f6e480b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.063+0000 7f6e47400640 1 -- 172.21.15.131:0/689268357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e480b4240 con 0x7f6e4815d5e0 2024-04-28T12:38:40.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.064+0000 7f6e47400640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4815d5e0 0x7f6e481419c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6e4002ace0 tx=0x7f6e40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:40.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.064+0000 7f6e3ea00640 1 -- 172.21.15.131:0/689268357 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40036050 con 0x7f6e4815d5e0 2024-04-28T12:38:40.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.064+0000 7f6e3ea00640 1 -- 172.21.15.131:0/689268357 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e40042070 con 0x7f6e4815d5e0 2024-04-28T12:38:40.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.064+0000 7f6e4da00640 1 -- 172.21.15.131:0/689268357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e480b44e0 con 0x7f6e4815d5e0 2024-04-28T12:38:40.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.064+0000 7f6e3ea00640 1 -- 172.21.15.131:0/689268357 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40009510 con 0x7f6e4815d5e0 2024-04-28T12:38:40.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.064+0000 7f6e4da00640 1 -- 172.21.15.131:0/689268357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e480b4aa0 con 0x7f6e4815d5e0 2024-04-28T12:38:40.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.065+0000 7f6e4da00640 1 -- 172.21.15.131:0/689268357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e48069350 con 0x7f6e4815d5e0 2024-04-28T12:38:40.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.066+0000 7f6e3ea00640 1 -- 172.21.15.131:0/689268357 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e40007c50 con 0x7f6e4815d5e0 2024-04-28T12:38:40.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.067+0000 7f6e3ea00640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6e2806c940 0x7f6e2806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.067+0000 7f6e3ea00640 1 -- 172.21.15.131:0/689268357 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6e400acda0 con 0x7f6e4815d5e0 2024-04-28T12:38:40.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.068+0000 7f6e46a00640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6e2806c940 0x7f6e2806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.069+0000 7f6e46a00640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6e2806c940 0x7f6e2806ee00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6e38004ea0 tx=0x7f6e38009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:40.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.072+0000 7f6e3ea00640 1 -- 172.21.15.131:0/689268357 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6e40045360 con 0x7f6e4815d5e0 2024-04-28T12:38:40.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.257+0000 7f6e4da00640 1 -- 172.21.15.131:0/689268357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6e480b5cf0 con 0x7f6e4815d5e0 2024-04-28T12:38:40.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.258+0000 7f6e3ea00640 1 -- 172.21.15.131:0/689268357 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f6e4007ae40 con 0x7f6e4815d5e0 2024-04-28T12:38:40.254 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:40.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.260+0000 7f6e4da00640 1 -- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6e2806c940 msgr2=0x7f6e2806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.260+0000 7f6e4da00640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6e2806c940 0x7f6e2806ee00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6e38004ea0 tx=0x7f6e38009040 comp rx=0 tx=0).stop 2024-04-28T12:38:40.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.260+0000 7f6e4da00640 1 -- 172.21.15.131:0/689268357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4815d5e0 msgr2=0x7f6e481419c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.260+0000 7f6e4da00640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4815d5e0 0x7f6e481419c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6e4002ace0 tx=0x7f6e40004a40 comp rx=0 tx=0).stop 2024-04-28T12:38:40.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.260+0000 7f6e4da00640 1 -- 172.21.15.131:0/689268357 shutdown_connections 2024-04-28T12:38:40.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.260+0000 7f6e4da00640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6e2806c940 0x7f6e2806ee00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.260+0000 7f6e4da00640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e4815d5e0 0x7f6e481419c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.260+0000 7f6e4da00640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e48145d80 0x7f6e481f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.260+0000 7f6e4da00640 1 --2- 172.21.15.131:0/689268357 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e48072a90 0x7f6e480b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.260+0000 7f6e4da00640 1 -- 172.21.15.131:0/689268357 >> 172.21.15.131:0/689268357 conn(0x7f6e480b2900 msgr2=0x7f6e480d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:40.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.260+0000 7f6e4da00640 1 -- 172.21.15.131:0/689268357 shutdown_connections 2024-04-28T12:38:40.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.260+0000 7f6e4da00640 1 -- 172.21.15.131:0/689268357 wait complete. 2024-04-28T12:38:40.276 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:40.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.461+0000 7f2593e00640 1 Processor -- start 2024-04-28T12:38:40.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.461+0000 7f2593e00640 1 -- start start 2024-04-28T12:38:40.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.463+0000 7f2593e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f259415a7e0 0x7f25940b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.463+0000 7f2593e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f259415d9c0 0x7f25940b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.463+0000 7f2593e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25940b3900 con 0x7f25940b5410 2024-04-28T12:38:40.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.463+0000 7f2593e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f259415abc0 con 0x7f259415d9c0 2024-04-28T12:38:40.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.463+0000 7f2593e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f25940b3b70 con 0x7f259415a7e0 2024-04-28T12:38:40.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.463+0000 7f2593400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f259415d9c0 0x7f25940b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.463+0000 7f2593400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f259415d9c0 0x7f25940b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37744/0 (socket says 172.21.15.131:37744) 2024-04-28T12:38:40.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.463+0000 7f2593400640 1 -- 172.21.15.131:0/3817587050 learned_addr learned my addr 172.21.15.131:0/3817587050 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:40.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.463+0000 7f2592a00640 1 --2- 172.21.15.131:0/3817587050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f259415a7e0 0x7f25940b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.464+0000 7f2593400640 1 -- 172.21.15.131:0/3817587050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f259415a7e0 msgr2=0x7f25940b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:40.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.464+0000 7f2593400640 1 --2- 172.21.15.131:0/3817587050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f259415a7e0 0x7f25940b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.464+0000 7f2593400640 1 -- 172.21.15.131:0/3817587050 >> v1:172.21.15.157:6789/0 conn(0x7f25940b5410 legacy=0x7f259415d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:40.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.464+0000 7f2593400640 1 -- 172.21.15.131:0/3817587050 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25940b3e50 con 0x7f259415d9c0 2024-04-28T12:38:40.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.465+0000 7f2593400640 1 --2- 172.21.15.131:0/3817587050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259415d9c0 0x7f25940b5890 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f258c009e50 tx=0x7f258c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6431ec0502367777 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:40.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.465+0000 7f2591600640 1 -- 172.21.15.131:0/3817587050 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c036070 con 0x7f259415d9c0 2024-04-28T12:38:40.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.465+0000 7f2591600640 1 -- 172.21.15.131:0/3817587050 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f258c031600 con 0x7f259415d9c0 2024-04-28T12:38:40.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.465+0000 7f2591600640 1 -- 172.21.15.131:0/3817587050 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c0319d0 con 0x7f259415d9c0 2024-04-28T12:38:40.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.466+0000 7f2593e00640 1 -- 172.21.15.131:0/3817587050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259415d9c0 msgr2=0x7f25940b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.466+0000 7f2593e00640 1 --2- 172.21.15.131:0/3817587050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259415d9c0 0x7f25940b5890 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f258c009e50 tx=0x7f258c02a710 comp rx=0 tx=0).stop 2024-04-28T12:38:40.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.466+0000 7f2593e00640 1 -- 172.21.15.131:0/3817587050 shutdown_connections 2024-04-28T12:38:40.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.466+0000 7f2593e00640 1 --2- 172.21.15.131:0/3817587050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259415d9c0 0x7f25940b5890 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.466+0000 7f2593e00640 1 --2- 172.21.15.131:0/3817587050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f259415a7e0 0x7f25940b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.466+0000 7f2593e00640 1 -- 172.21.15.131:0/3817587050 >> 172.21.15.131:0/3817587050 conn(0x7f2594068090 msgr2=0x7f25940684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:40.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.466+0000 7f2593e00640 1 -- 172.21.15.131:0/3817587050 shutdown_connections 2024-04-28T12:38:40.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.466+0000 7f2593e00640 1 -- 172.21.15.131:0/3817587050 wait complete. 2024-04-28T12:38:40.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.467+0000 7f2593e00640 1 Processor -- start 2024-04-28T12:38:40.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.468+0000 7f2593e00640 1 -- start start 2024-04-28T12:38:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.468+0000 7f2593e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25940b5410 0x7f259414d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.468+0000 7f2593e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259415a7e0 0x7f25941f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.468+0000 7f2593e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f259415d9c0 0x7f25941f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.468+0000 7f2593e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25940b4b90 con 0x7f259415a7e0 2024-04-28T12:38:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.468+0000 7f2593e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f259415e010 con 0x7f25940b5410 2024-04-28T12:38:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.468+0000 7f2593e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f259415e2e0 con 0x7f259415d9c0 2024-04-28T12:38:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.469+0000 7f2592000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259415a7e0 0x7f25941f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.469+0000 7f2592000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259415a7e0 0x7f25941f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37760/0 (socket says 172.21.15.131:37760) 2024-04-28T12:38:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.469+0000 7f2592000640 1 -- 172.21.15.131:0/2033312328 learned_addr learned my addr 172.21.15.131:0/2033312328 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.469+0000 7f2592a00640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25940b5410 0x7f259414d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.469+0000 7f2592000640 1 -- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f259415d9c0 msgr2=0x7f25941f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.469+0000 7f2592000640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f259415d9c0 0x7f25941f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.469+0000 7f2592000640 1 -- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25940b5410 msgr2=0x7f259414d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.469+0000 7f2592000640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25940b5410 0x7f259414d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.469+0000 7f2592000640 1 -- 172.21.15.131:0/2033312328 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25941f54d0 con 0x7f259415a7e0 2024-04-28T12:38:40.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.469+0000 7f2592000640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259415a7e0 0x7f25941f89f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f258000c400 tx=0x7f258000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:40.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.470+0000 7f258aa00640 1 -- 172.21.15.131:0/2033312328 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800052d0 con 0x7f259415a7e0 2024-04-28T12:38:40.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.470+0000 7f258aa00640 1 -- 172.21.15.131:0/2033312328 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2580005490 con 0x7f259415a7e0 2024-04-28T12:38:40.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.470+0000 7f2593e00640 1 -- 172.21.15.131:0/2033312328 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25941f57b0 con 0x7f259415a7e0 2024-04-28T12:38:40.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.470+0000 7f258aa00640 1 -- 172.21.15.131:0/2033312328 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580007260 con 0x7f259415a7e0 2024-04-28T12:38:40.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.470+0000 7f2593e00640 1 -- 172.21.15.131:0/2033312328 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25941f8f30 con 0x7f259415a7e0 2024-04-28T12:38:40.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.471+0000 7f2593e00640 1 -- 172.21.15.131:0/2033312328 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25940506c0 con 0x7f259415a7e0 2024-04-28T12:38:40.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.472+0000 7f258aa00640 1 -- 172.21.15.131:0/2033312328 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2580007400 con 0x7f259415a7e0 2024-04-28T12:38:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.473+0000 7f258aa00640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f257406c8f0 0x7f257406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.473+0000 7f258aa00640 1 -- 172.21.15.131:0/2033312328 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f258008b4a0 con 0x7f259415a7e0 2024-04-28T12:38:40.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.473+0000 7f2592a00640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f257406c8f0 0x7f257406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.475+0000 7f2592a00640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f257406c8f0 0x7f257406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2584006fd0 tx=0x7f2584008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:40.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.477+0000 7f258aa00640 1 -- 172.21.15.131:0/2033312328 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f25800594d0 con 0x7f259415a7e0 2024-04-28T12:38:40.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.661+0000 7f2593e00640 1 -- 172.21.15.131:0/2033312328 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f25940d1700 con 0x7f259415a7e0 2024-04-28T12:38:40.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.662+0000 7f258aa00640 1 -- 172.21.15.131:0/2033312328 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f2580058dd0 con 0x7f259415a7e0 2024-04-28T12:38:40.659 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:40.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.664+0000 7f2593e00640 1 -- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f257406c8f0 msgr2=0x7f257406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.664+0000 7f2593e00640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f257406c8f0 0x7f257406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2584006fd0 tx=0x7f2584008040 comp rx=0 tx=0).stop 2024-04-28T12:38:40.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.664+0000 7f2593e00640 1 -- 172.21.15.131:0/2033312328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259415a7e0 msgr2=0x7f25941f89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.664+0000 7f2593e00640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259415a7e0 0x7f25941f89f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f258000c400 tx=0x7f258000c8c0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.664+0000 7f2593e00640 1 -- 172.21.15.131:0/2033312328 shutdown_connections 2024-04-28T12:38:40.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.664+0000 7f2593e00640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f257406c8f0 0x7f257406edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.664+0000 7f2593e00640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f259415d9c0 0x7f25941f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.664+0000 7f2593e00640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f259415a7e0 0x7f25941f89f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.664+0000 7f2593e00640 1 --2- 172.21.15.131:0/2033312328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25940b5410 0x7f259414d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.664+0000 7f2593e00640 1 -- 172.21.15.131:0/2033312328 >> 172.21.15.131:0/2033312328 conn(0x7f2594068090 msgr2=0x7f2594159510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:40.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.665+0000 7f2593e00640 1 -- 172.21.15.131:0/2033312328 shutdown_connections 2024-04-28T12:38:40.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.665+0000 7f2593e00640 1 -- 172.21.15.131:0/2033312328 wait complete. 2024-04-28T12:38:40.679 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T12:38:40.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.870+0000 7f7dede00640 1 Processor -- start 2024-04-28T12:38:40.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.870+0000 7f7dede00640 1 -- start start 2024-04-28T12:38:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.872+0000 7f7dede00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7de81628e0 0x7f7de8162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.872+0000 7f7dede00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7de80af720 0x7f7de80afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.872+0000 7f7dede00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7de8145710 con 0x7f7de8145330 2024-04-28T12:38:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.872+0000 7f7dede00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7de8163280 con 0x7f7de80af720 2024-04-28T12:38:40.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.872+0000 7f7dede00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7de8073dc0 con 0x7f7de81628e0 2024-04-28T12:38:40.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.872+0000 7f7deca00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7de8145330 0x7f7de80b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49284/0 (socket says 172.21.15.131:49284) 2024-04-28T12:38:40.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.872+0000 7f7deca00640 1 -- 172.21.15.131:0/1157616976 learned_addr learned my addr 172.21.15.131:0/1157616976 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:40.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.872+0000 7f7de7e00640 1 --2- 172.21.15.131:0/1157616976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7de81628e0 0x7f7de8162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.872+0000 7f7ded400640 1 --2- 172.21.15.131:0/1157616976 >> v2:172.21.15.157:3300/0 conn(0x7f7de80af720 0x7f7de80afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.872+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3609305849 0 0) 0x7f7de8145710 con 0x7f7de8145330 2024-04-28T12:38:40.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7dd0003520 con 0x7f7de8145330 2024-04-28T12:38:40.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1370624408 0 0) 0x7f7dd0003520 con 0x7f7de8145330 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7de81628e0 msgr2=0x7f7de8162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 --2- 172.21.15.131:0/1157616976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7de81628e0 0x7f7de8162d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 >> v2:172.21.15.157:3300/0 conn(0x7f7de80af720 msgr2=0x7f7de80afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 --2- 172.21.15.131:0/1157616976 >> v2:172.21.15.157:3300/0 conn(0x7f7de80af720 0x7f7de80afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de80ae960 con 0x7f7de8145330 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f7ddc001670 con 0x7f7de8145330 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 >> v1:172.21.15.131:6789/0 conn(0x7f7de8145330 legacy=0x7f7de80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 --2- 172.21.15.131:0/1157616976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dd00039b0 0x7f7dd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 --2- 172.21.15.131:0/1157616976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd0004800 0x7f7dd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 --2- 172.21.15.131:0/1157616976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7de81628e0 0x7f7dd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7de8073dc0 con 0x7f7dd00039b0 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd000b9c0 con 0x7f7dd0004800 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd000bb40 con 0x7f7de81628e0 2024-04-28T12:38:40.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.873+0000 7f7deca00640 1 --2- 172.21.15.131:0/1157616976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dd00039b0 0x7f7dd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.874+0000 7f7de7e00640 1 --2- 172.21.15.131:0/1157616976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd0004800 0x7f7dd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.874+0000 7f7deca00640 1 -- 172.21.15.131:0/1157616976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7de81628e0 msgr2=0x7f7dd00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.874+0000 7f7deca00640 1 --2- 172.21.15.131:0/1157616976 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7de81628e0 0x7f7dd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.874+0000 7f7deca00640 1 -- 172.21.15.131:0/1157616976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd0004800 msgr2=0x7f7dd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.874+0000 7f7deca00640 1 --2- 172.21.15.131:0/1157616976 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd0004800 0x7f7dd0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.874+0000 7f7deca00640 1 -- 172.21.15.131:0/1157616976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd0003520 con 0x7f7dd00039b0 2024-04-28T12:38:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.875+0000 7f7deca00640 1 --2- 172.21.15.131:0/1157616976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dd00039b0 0x7f7dd0003dd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00c290 tx=0x7f7dd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220bf77577170d0a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.875+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc007d50 con 0x7f7dd00039b0 2024-04-28T12:38:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.875+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ddc005af0 con 0x7f7dd00039b0 2024-04-28T12:38:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.875+0000 7f7de7400640 1 -- 172.21.15.131:0/1157616976 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc007070 con 0x7f7dd00039b0 2024-04-28T12:38:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.875+0000 7f7dede00640 1 -- 172.21.15.131:0/1157616976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dd00039b0 msgr2=0x7f7dd0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.875+0000 7f7dede00640 1 --2- 172.21.15.131:0/1157616976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dd00039b0 0x7f7dd0003dd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00c290 tx=0x7f7dd0005830 comp rx=0 tx=0).stop 2024-04-28T12:38:40.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.876+0000 7f7ded400640 1 -- 172.21.15.131:0/1157616976 reap_dead start 2024-04-28T12:38:40.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.876+0000 7f7dede00640 1 -- 172.21.15.131:0/1157616976 shutdown_connections 2024-04-28T12:38:40.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.876+0000 7f7dede00640 1 -- 172.21.15.131:0/1157616976 >> 172.21.15.131:0/1157616976 conn(0x7f7de8068090 msgr2=0x7f7de8068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:40.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.876+0000 7f7dede00640 1 -- 172.21.15.131:0/1157616976 shutdown_connections 2024-04-28T12:38:40.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.876+0000 7f7dede00640 1 -- 172.21.15.131:0/1157616976 wait complete. 2024-04-28T12:38:40.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.877+0000 7f7dede00640 1 Processor -- start 2024-04-28T12:38:40.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.877+0000 7f7dede00640 1 -- start start 2024-04-28T12:38:40.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.878+0000 7f7dede00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dd00039b0 0x7f7de81350c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.878+0000 7f7dede00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de80b3280 0x7f7de8161550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.878+0000 7f7dede00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de8161a90 0x7f7de80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.878+0000 7f7dede00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7de80af200 con 0x7f7de8161a90 2024-04-28T12:38:40.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.878+0000 7f7dede00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7de80aef20 con 0x7f7de80b3280 2024-04-28T12:38:40.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.878+0000 7f7dede00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7de80b00f0 con 0x7f7dd00039b0 2024-04-28T12:38:40.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.879+0000 7f7ded400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de8161a90 0x7f7de80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.879+0000 7f7ded400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de8161a90 0x7f7de80b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42104/0 (socket says 172.21.15.131:42104) 2024-04-28T12:38:40.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.879+0000 7f7ded400640 1 -- 172.21.15.131:0/1755055714 learned_addr learned my addr 172.21.15.131:0/1755055714 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:40.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.879+0000 7f7ded400640 1 -- 172.21.15.131:0/1755055714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dd00039b0 msgr2=0x7f7de81350c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.879+0000 7f7ded400640 1 --2- 172.21.15.131:0/1755055714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dd00039b0 0x7f7de81350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.879+0000 7f7ded400640 1 -- 172.21.15.131:0/1755055714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de80b3280 msgr2=0x7f7de8161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:40.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.879+0000 7f7ded400640 1 --2- 172.21.15.131:0/1755055714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de80b3280 0x7f7de8161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:40.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.879+0000 7f7ded400640 1 -- 172.21.15.131:0/1755055714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de81f85d0 con 0x7f7de8161a90 2024-04-28T12:38:40.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.879+0000 7f7ded400640 1 --2- 172.21.15.131:0/1755055714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de8161a90 0x7f7de80b15c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7de806fd10 tx=0x7f7de000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:40.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.879+0000 7f7de4c00640 1 -- 172.21.15.131:0/1755055714 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de000bd10 con 0x7f7de8161a90 2024-04-28T12:38:40.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.879+0000 7f7de4c00640 1 -- 172.21.15.131:0/1755055714 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de001a070 con 0x7f7de8161a90 2024-04-28T12:38:40.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.879+0000 7f7dede00640 1 -- 172.21.15.131:0/1755055714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de81f88b0 con 0x7f7de8161a90 2024-04-28T12:38:40.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.880+0000 7f7de4c00640 1 -- 172.21.15.131:0/1755055714 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0014640 con 0x7f7de8161a90 2024-04-28T12:38:40.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.880+0000 7f7dede00640 1 -- 172.21.15.131:0/1755055714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de81f8e70 con 0x7f7de8161a90 2024-04-28T12:38:40.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.881+0000 7f7dede00640 1 -- 172.21.15.131:0/1755055714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de8161f10 con 0x7f7de8161a90 2024-04-28T12:38:40.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.882+0000 7f7de4c00640 1 -- 172.21.15.131:0/1755055714 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7de0005d10 con 0x7f7de8161a90 2024-04-28T12:38:40.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.883+0000 7f7de4c00640 1 --2- 172.21.15.131:0/1755055714 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7dc806c920 0x7f7dc806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:40.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.883+0000 7f7de4c00640 1 -- 172.21.15.131:0/1755055714 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7de008c080 con 0x7f7de8161a90 2024-04-28T12:38:40.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.883+0000 7f7deca00640 1 --2- 172.21.15.131:0/1755055714 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7dc806c920 0x7f7dc806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:40.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.884+0000 7f7deca00640 1 --2- 172.21.15.131:0/1755055714 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7dc806c920 0x7f7dc806ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc033000 tx=0x7f7ddc007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:40.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:40.887+0000 7f7de4c00640 1 -- 172.21.15.131:0/1755055714 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7de005a010 con 0x7f7de8161a90 2024-04-28T12:38:41.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.095+0000 7f7dede00640 1 -- 172.21.15.131:0/1755055714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7de8069350 con 0x7f7de8161a90 2024-04-28T12:38:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.096+0000 7f7de4c00640 1 -- 172.21.15.131:0/1755055714 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f7de0059910 con 0x7f7de8161a90 2024-04-28T12:38:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-04-28T12:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.099+0000 7f7dede00640 1 -- 172.21.15.131:0/1755055714 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7dc806c920 msgr2=0x7f7dc806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.099+0000 7f7dede00640 1 --2- 172.21.15.131:0/1755055714 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7dc806c920 0x7f7dc806ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc033000 tx=0x7f7ddc007000 comp rx=0 tx=0).stop 2024-04-28T12:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.099+0000 7f7dede00640 1 -- 172.21.15.131:0/1755055714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de8161a90 msgr2=0x7f7de80b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.099+0000 7f7dede00640 1 --2- 172.21.15.131:0/1755055714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de8161a90 0x7f7de80b15c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7de806fd10 tx=0x7f7de000b480 comp rx=0 tx=0).stop 2024-04-28T12:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.099+0000 7f7dede00640 1 -- 172.21.15.131:0/1755055714 shutdown_connections 2024-04-28T12:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.099+0000 7f7dede00640 1 --2- 172.21.15.131:0/1755055714 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7dc806c920 0x7f7dc806ede0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.099+0000 7f7dede00640 1 --2- 172.21.15.131:0/1755055714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7de8161a90 0x7f7de80b15c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.099+0000 7f7dede00640 1 --2- 172.21.15.131:0/1755055714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de80b3280 0x7f7de8161550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.099+0000 7f7dede00640 1 --2- 172.21.15.131:0/1755055714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dd00039b0 0x7f7de81350c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.099+0000 7f7dede00640 1 -- 172.21.15.131:0/1755055714 >> 172.21.15.131:0/1755055714 conn(0x7f7de8068090 msgr2=0x7f7de8068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.099+0000 7f7dede00640 1 -- 172.21.15.131:0/1755055714 shutdown_connections 2024-04-28T12:38:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.099+0000 7f7dede00640 1 -- 172.21.15.131:0/1755055714 wait complete. 2024-04-28T12:38:41.114 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-28T12:38:41.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:41.114 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-28T12:38:41.213 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:38:41.213 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:38:41.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:41.213 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-28T12:38:41.272 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:38:41.272 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:38:41.329 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:38:41.330 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:38:41.381 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:41.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.577+0000 7fdca9600640 1 Processor -- start 2024-04-28T12:38:41.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.577+0000 7fdca9600640 1 -- start start 2024-04-28T12:38:41.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.579+0000 7fdca9600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdca40cd180 0x7fdca4072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:41.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.579+0000 7fdca9600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdca40cd560 con 0x7fdca40ce480 2024-04-28T12:38:41.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.579+0000 7fdca9600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdca4072d00 con 0x7fdca415dc50 2024-04-28T12:38:41.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.579+0000 7fdca9600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fdca4072fe0 con 0x7fdca40cd180 2024-04-28T12:38:41.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.579+0000 7fdca3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdca40cd180 0x7fdca4072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:41.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.579+0000 7fdca3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdca40cd180 0x7fdca4072620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42112/0 (socket says 172.21.15.131:42112) 2024-04-28T12:38:41.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.579+0000 7fdca3e00640 1 -- 172.21.15.131:0/1022501749 learned_addr learned my addr 172.21.15.131:0/1022501749 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:41.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.580+0000 7fdca3e00640 1 -- 172.21.15.131:0/1022501749 >> v1:172.21.15.157:6789/0 conn(0x7fdca415dc50 legacy=0x7fdca40cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:41.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.580+0000 7fdca3e00640 1 -- 172.21.15.131:0/1022501749 >> v1:172.21.15.131:6789/0 conn(0x7fdca40ce480 legacy=0x7fdca406dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:41.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.580+0000 7fdca3e00640 1 -- 172.21.15.131:0/1022501749 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca40732c0 con 0x7fdca40cd180 2024-04-28T12:38:41.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.580+0000 7fdca3400640 1 -- 172.21.15.131:0/1022501749 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670334144 0 0) 0x7fdca40cd560 con 0x7fdca40ce480 2024-04-28T12:38:41.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.581+0000 7fdca3e00640 1 --2- 172.21.15.131:0/1022501749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cd180 0x7fdca4072620 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009870 tx=0x7fdc9402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850933baaa23b76a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:41.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.581+0000 7fdca3400640 1 -- 172.21.15.131:0/1022501749 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94037070 con 0x7fdca40cd180 2024-04-28T12:38:41.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.581+0000 7fdca3400640 1 -- 172.21.15.131:0/1022501749 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc9402ae50 con 0x7fdca40cd180 2024-04-28T12:38:41.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.582+0000 7fdca3400640 1 -- 172.21.15.131:0/1022501749 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94032860 con 0x7fdca40cd180 2024-04-28T12:38:41.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.582+0000 7fdca9600640 1 -- 172.21.15.131:0/1022501749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cd180 msgr2=0x7fdca4072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:41.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.582+0000 7fdca9600640 1 --2- 172.21.15.131:0/1022501749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cd180 0x7fdca4072620 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009870 tx=0x7fdc9402a300 comp rx=0 tx=0).stop 2024-04-28T12:38:41.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.582+0000 7fdca9600640 1 -- 172.21.15.131:0/1022501749 shutdown_connections 2024-04-28T12:38:41.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.582+0000 7fdca9600640 1 --2- 172.21.15.131:0/1022501749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cd180 0x7fdca4072620 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.582+0000 7fdca9600640 1 -- 172.21.15.131:0/1022501749 >> 172.21.15.131:0/1022501749 conn(0x7fdca4068090 msgr2=0x7fdca4068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:41.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.582+0000 7fdca9600640 1 -- 172.21.15.131:0/1022501749 shutdown_connections 2024-04-28T12:38:41.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.582+0000 7fdca9600640 1 -- 172.21.15.131:0/1022501749 wait complete. 2024-04-28T12:38:41.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.584+0000 7fdca9600640 1 Processor -- start 2024-04-28T12:38:41.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.584+0000 7fdca9600640 1 -- start start 2024-04-28T12:38:41.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.585+0000 7fdca9600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cd180 0x7fdca413dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.585+0000 7fdca9600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca40ce480 0x7fdca415e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.585+0000 7fdca9600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca415dc50 0x7fdca415e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.585+0000 7fdca9600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdca415e0d0 con 0x7fdca40cd180 2024-04-28T12:38:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.585+0000 7fdca9600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdca4073da0 con 0x7fdca415dc50 2024-04-28T12:38:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.585+0000 7fdca9600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdca406e1d0 con 0x7fdca40ce480 2024-04-28T12:38:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.585+0000 7fdca3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cd180 0x7fdca413dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.585+0000 7fdca3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cd180 0x7fdca413dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42124/0 (socket says 172.21.15.131:42124) 2024-04-28T12:38:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.586+0000 7fdca3e00640 1 -- 172.21.15.131:0/2596077393 learned_addr learned my addr 172.21.15.131:0/2596077393 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.586+0000 7fdca8c00640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca415dc50 0x7fdca415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.586+0000 7fdca3e00640 1 -- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca40ce480 msgr2=0x7fdca415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.586+0000 7fdca3e00640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca40ce480 0x7fdca415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.586+0000 7fdca3e00640 1 -- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca415dc50 msgr2=0x7fdca415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:41.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.586+0000 7fdca3e00640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca415dc50 0x7fdca415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.586+0000 7fdca3e00640 1 -- 172.21.15.131:0/2596077393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca41548e0 con 0x7fdca40cd180 2024-04-28T12:38:41.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.586+0000 7fdca8c00640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca415dc50 0x7fdca415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:41.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.586+0000 7fdca3e00640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cd180 0x7fdca413dad0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009870 tx=0x7fdc940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.587+0000 7fdca0c00640 1 -- 172.21.15.131:0/2596077393 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94037040 con 0x7fdca40cd180 2024-04-28T12:38:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.587+0000 7fdca0c00640 1 -- 172.21.15.131:0/2596077393 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc94004230 con 0x7fdca40cd180 2024-04-28T12:38:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.587+0000 7fdca0c00640 1 -- 172.21.15.131:0/2596077393 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94032e40 con 0x7fdca40cd180 2024-04-28T12:38:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.587+0000 7fdca9600640 1 -- 172.21.15.131:0/2596077393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca4142470 con 0x7fdca40cd180 2024-04-28T12:38:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.587+0000 7fdca9600640 1 -- 172.21.15.131:0/2596077393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca413ee40 con 0x7fdca40cd180 2024-04-28T12:38:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.588+0000 7fdca9600640 1 -- 172.21.15.131:0/2596077393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca4069350 con 0x7fdca40cd180 2024-04-28T12:38:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.589+0000 7fdca0c00640 1 -- 172.21.15.131:0/2596077393 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc9403a070 con 0x7fdca40cd180 2024-04-28T12:38:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.590+0000 7fdca0c00640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdc8006c8f0 0x7fdc8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.590+0000 7fdca0c00640 1 -- 172.21.15.131:0/2596077393 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdc940ac660 con 0x7fdca40cd180 2024-04-28T12:38:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.590+0000 7fdc9b400640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdc8006c8f0 0x7fdc8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:41.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.592+0000 7fdc9b400640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdc8006c8f0 0x7fdc8006edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c000f60 tx=0x7fdc8c0079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:41.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.594+0000 7fdca0c00640 1 -- 172.21.15.131:0/2596077393 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc94041020 con 0x7fdca40cd180 2024-04-28T12:38:41.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.799+0000 7fdca9600640 1 -- 172.21.15.131:0/2596077393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdca40d0670 con 0x7fdca40cd180 2024-04-28T12:38:41.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.803+0000 7fdca0c00640 1 -- 172.21.15.131:0/2596077393 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7fdc9407a720 con 0x7fdca40cd180 2024-04-28T12:38:41.800 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:41.800 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:41.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.806+0000 7fdca9600640 1 -- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdc8006c8f0 msgr2=0x7fdc8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:41.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.806+0000 7fdca9600640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdc8006c8f0 0x7fdc8006edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c000f60 tx=0x7fdc8c0079b0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.806+0000 7fdca9600640 1 -- 172.21.15.131:0/2596077393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cd180 msgr2=0x7fdca413dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:41.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.806+0000 7fdca9600640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cd180 0x7fdca413dad0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009870 tx=0x7fdc940046c0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.806+0000 7fdca9600640 1 -- 172.21.15.131:0/2596077393 shutdown_connections 2024-04-28T12:38:41.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.806+0000 7fdca9600640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdc8006c8f0 0x7fdc8006edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.806+0000 7fdca9600640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca415dc50 0x7fdca415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.806+0000 7fdca9600640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca40ce480 0x7fdca415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.806+0000 7fdca9600640 1 --2- 172.21.15.131:0/2596077393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cd180 0x7fdca413dad0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:41.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.806+0000 7fdca9600640 1 -- 172.21.15.131:0/2596077393 >> 172.21.15.131:0/2596077393 conn(0x7fdca4068090 msgr2=0x7fdca406e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:41.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.806+0000 7fdca9600640 1 -- 172.21.15.131:0/2596077393 shutdown_connections 2024-04-28T12:38:41.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:41.806+0000 7fdca9600640 1 -- 172.21.15.131:0/2596077393 wait complete. 2024-04-28T12:38:41.822 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:41.823 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:42.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.033+0000 7fcf09c00640 1 Processor -- start 2024-04-28T12:38:42.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.033+0000 7fcf09c00640 1 -- start start 2024-04-28T12:38:42.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.034+0000 7fcf09c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf0415a7e0 0x7fcf040b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.034+0000 7fcf09c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf040b5410 0x7fcf040b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.034+0000 7fcf09c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf040b3900 con 0x7fcf0415d2b0 2024-04-28T12:38:42.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.034+0000 7fcf09c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcf0415abc0 con 0x7fcf0415a7e0 2024-04-28T12:38:42.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.034+0000 7fcf09c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcf040b3b70 con 0x7fcf040b5410 2024-04-28T12:38:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.035+0000 7fcf03e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf0415a7e0 0x7fcf040b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.035+0000 7fcf09200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcf0415d2b0 0x7fcf0415d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49306/0 (socket says 172.21.15.131:49306) 2024-04-28T12:38:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.035+0000 7fcf09200640 1 -- 172.21.15.131:0/2057079949 learned_addr learned my addr 172.21.15.131:0/2057079949 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.035+0000 7fcefb400640 1 --2- 172.21.15.131:0/2057079949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf040b5410 0x7fcf040b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.035+0000 7fcf03400640 1 -- 172.21.15.131:0/2057079949 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1093304804 0 0) 0x7fcf040b3900 con 0x7fcf0415d2b0 2024-04-28T12:38:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.035+0000 7fcf03400640 1 -- 172.21.15.131:0/2057079949 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcee8003520 con 0x7fcf0415d2b0 2024-04-28T12:38:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.036+0000 7fcf03e00640 1 -- 172.21.15.131:0/2057079949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf040b5410 msgr2=0x7fcf040b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.036+0000 7fcf03e00640 1 --2- 172.21.15.131:0/2057079949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf040b5410 0x7fcf040b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.036+0000 7fcf03e00640 1 -- 172.21.15.131:0/2057079949 >> v1:172.21.15.131:6789/0 conn(0x7fcf0415d2b0 legacy=0x7fcf0415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.036+0000 7fcf03e00640 1 -- 172.21.15.131:0/2057079949 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf040b3e50 con 0x7fcf0415a7e0 2024-04-28T12:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.037+0000 7fcf03e00640 1 --2- 172.21.15.131:0/2057079949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf0415a7e0 0x7fcf040b4650 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcef40097a0 tx=0x7fcef402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9ccb91d7814b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.037+0000 7fcf03400640 1 -- 172.21.15.131:0/2057079949 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4037070 con 0x7fcf0415a7e0 2024-04-28T12:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.037+0000 7fcf03400640 1 -- 172.21.15.131:0/2057079949 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef4032440 con 0x7fcf0415a7e0 2024-04-28T12:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.037+0000 7fcf03400640 1 -- 172.21.15.131:0/2057079949 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4032810 con 0x7fcf0415a7e0 2024-04-28T12:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.037+0000 7fcf09c00640 1 -- 172.21.15.131:0/2057079949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf0415a7e0 msgr2=0x7fcf040b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.037+0000 7fcf09c00640 1 --2- 172.21.15.131:0/2057079949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf0415a7e0 0x7fcf040b4650 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcef40097a0 tx=0x7fcef402a180 comp rx=0 tx=0).stop 2024-04-28T12:38:42.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.038+0000 7fcf09c00640 1 -- 172.21.15.131:0/2057079949 shutdown_connections 2024-04-28T12:38:42.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.038+0000 7fcf09c00640 1 --2- 172.21.15.131:0/2057079949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf040b5410 0x7fcf040b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.038+0000 7fcf09c00640 1 --2- 172.21.15.131:0/2057079949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf0415a7e0 0x7fcf040b4650 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.038+0000 7fcf09c00640 1 -- 172.21.15.131:0/2057079949 >> 172.21.15.131:0/2057079949 conn(0x7fcf04068090 msgr2=0x7fcf040684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:42.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.038+0000 7fcf09c00640 1 -- 172.21.15.131:0/2057079949 shutdown_connections 2024-04-28T12:38:42.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.038+0000 7fcf09c00640 1 -- 172.21.15.131:0/2057079949 wait complete. 2024-04-28T12:38:42.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.039+0000 7fcf09c00640 1 Processor -- start 2024-04-28T12:38:42.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.040+0000 7fcf09c00640 1 -- start start 2024-04-28T12:38:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.040+0000 7fcf09c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf040b5410 0x7fcf041f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.040+0000 7fcf09c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf0415a7e0 0x7fcf041e5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcf09c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf0415d2b0 0x7fcf0415bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcf09c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcf040b4b90 con 0x7fcf0415a7e0 2024-04-28T12:38:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcf09c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0415e010 con 0x7fcf0415d2b0 2024-04-28T12:38:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcf09c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0415e2e0 con 0x7fcf040b5410 2024-04-28T12:38:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcefb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf0415a7e0 0x7fcf041e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcefb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf0415a7e0 0x7fcf041e5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42156/0 (socket says 172.21.15.131:42156) 2024-04-28T12:38:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcefb400640 1 -- 172.21.15.131:0/2290533999 learned_addr learned my addr 172.21.15.131:0/2290533999 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcefb400640 1 -- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf040b5410 msgr2=0x7fcf041f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcefb400640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf040b5410 0x7fcf041f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcf09200640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf0415d2b0 0x7fcf0415bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcefb400640 1 -- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf0415d2b0 msgr2=0x7fcf0415bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcefb400640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf0415d2b0 0x7fcf0415bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcefb400640 1 -- 172.21.15.131:0/2290533999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf040b02f0 con 0x7fcf0415a7e0 2024-04-28T12:38:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.041+0000 7fcf09200640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf0415d2b0 0x7fcf0415bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.042+0000 7fcefb400640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf0415a7e0 0x7fcf041e5240 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fceec00e9b0 tx=0x7fceec00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.042+0000 7fcf00c00640 1 -- 172.21.15.131:0/2290533999 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec017070 con 0x7fcf0415a7e0 2024-04-28T12:38:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.042+0000 7fcf00c00640 1 -- 172.21.15.131:0/2290533999 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceec00c9a0 con 0x7fcf0415a7e0 2024-04-28T12:38:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.042+0000 7fcf09c00640 1 -- 172.21.15.131:0/2290533999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf040b05d0 con 0x7fcf0415a7e0 2024-04-28T12:38:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.042+0000 7fcf00c00640 1 -- 172.21.15.131:0/2290533999 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec00cd90 con 0x7fcf0415a7e0 2024-04-28T12:38:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.042+0000 7fcf09c00640 1 -- 172.21.15.131:0/2290533999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf040b0b90 con 0x7fcf0415a7e0 2024-04-28T12:38:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.043+0000 7fcf09c00640 1 -- 172.21.15.131:0/2290533999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf04069d80 con 0x7fcf0415a7e0 2024-04-28T12:38:42.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.044+0000 7fcf00c00640 1 -- 172.21.15.131:0/2290533999 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fceec01a070 con 0x7fcf0415a7e0 2024-04-28T12:38:42.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.045+0000 7fcf00c00640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcee006c8f0 0x7fcee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.045+0000 7fcf00c00640 1 -- 172.21.15.131:0/2290533999 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fceec08c2f0 con 0x7fcf0415a7e0 2024-04-28T12:38:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.046+0000 7fcf03e00640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcee006c8f0 0x7fcee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.047+0000 7fcf03e00640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcee006c8f0 0x7fcee006edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcef4034000 tx=0x7fcef402c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:42.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.050+0000 7fcf00c00640 1 -- 172.21.15.131:0/2290533999 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf04069d80 con 0x7fcf0415a7e0 2024-04-28T12:38:42.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.251+0000 7fcf09c00640 1 -- 172.21.15.131:0/2290533999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf040506c0 con 0x7fcf0415a7e0 2024-04-28T12:38:42.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.251+0000 7fcf00c00640 1 -- 172.21.15.131:0/2290533999 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fceec05a310 con 0x7fcf0415a7e0 2024-04-28T12:38:42.290 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.254+0000 7fcf09c00640 1 -- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcee006c8f0 msgr2=0x7fcee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.254+0000 7fcf09c00640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcee006c8f0 0x7fcee006edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcef4034000 tx=0x7fcef402c000 comp rx=0 tx=0).stop 2024-04-28T12:38:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.254+0000 7fcf09c00640 1 -- 172.21.15.131:0/2290533999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf0415a7e0 msgr2=0x7fcf041e5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.254+0000 7fcf09c00640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf0415a7e0 0x7fcf041e5240 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fceec00e9b0 tx=0x7fceec00ee70 comp rx=0 tx=0).stop 2024-04-28T12:38:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.254+0000 7fcf09c00640 1 -- 172.21.15.131:0/2290533999 shutdown_connections 2024-04-28T12:38:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.254+0000 7fcf09c00640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcee006c8f0 0x7fcee006edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.254+0000 7fcf09c00640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcf0415d2b0 0x7fcf0415bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.254+0000 7fcf09c00640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf0415a7e0 0x7fcf041e5240 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.254+0000 7fcf09c00640 1 --2- 172.21.15.131:0/2290533999 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcf040b5410 0x7fcf041f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.254+0000 7fcf09c00640 1 -- 172.21.15.131:0/2290533999 >> 172.21.15.131:0/2290533999 conn(0x7fcf04068090 msgr2=0x7fcf040cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:42.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.255+0000 7fcf09c00640 1 -- 172.21.15.131:0/2290533999 shutdown_connections 2024-04-28T12:38:42.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.255+0000 7fcf09c00640 1 -- 172.21.15.131:0/2290533999 wait complete. 2024-04-28T12:38:42.292 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:42.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.502+0000 7fc5cf400640 1 Processor -- start 2024-04-28T12:38:42.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.502+0000 7fc5cf400640 1 -- start start 2024-04-28T12:38:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.504+0000 7fc5cf400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc5c815f1e0 0x7fc5c806e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.504+0000 7fc5cf400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc5c80b5ef0 0x7fc5c8072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.504+0000 7fc5cf400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c806e7b0 con 0x7fc5c8071ee0 2024-04-28T12:38:42.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.504+0000 7fc5cf400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c815f5c0 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.504+0000 7fc5cf400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c80729a0 con 0x7fc5c815f1e0 2024-04-28T12:38:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.504+0000 7fc5c7e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc5c8071ee0 0x7fc5c80b57e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49308/0 (socket says 172.21.15.131:49308) 2024-04-28T12:38:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.504+0000 7fc5c7e00640 1 -- 172.21.15.131:0/76363646 learned_addr learned my addr 172.21.15.131:0/76363646 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.505+0000 7fc5cd400640 1 --2- 172.21.15.131:0/76363646 >> v2:172.21.15.131:3300/0 conn(0x7fc5c80b5ef0 0x7fc5c8072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.505+0000 7fc5cca00640 1 --2- 172.21.15.131:0/76363646 >> v2:172.21.15.157:3300/0 conn(0x7fc5c815f1e0 0x7fc5c806e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.505+0000 7fc5c7400640 1 -- 172.21.15.131:0/76363646 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396419558 0 0) 0x7fc5c806e7b0 con 0x7fc5c8071ee0 2024-04-28T12:38:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.505+0000 7fc5cd400640 1 -- 172.21.15.131:0/76363646 >> v2:172.21.15.157:3300/0 conn(0x7fc5c815f1e0 msgr2=0x7fc5c806e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.505+0000 7fc5cd400640 1 --2- 172.21.15.131:0/76363646 >> v2:172.21.15.157:3300/0 conn(0x7fc5c815f1e0 0x7fc5c806e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.505+0000 7fc5cd400640 1 -- 172.21.15.131:0/76363646 >> v1:172.21.15.131:6789/0 conn(0x7fc5c8071ee0 legacy=0x7fc5c80b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.505+0000 7fc5cd400640 1 -- 172.21.15.131:0/76363646 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c8072c80 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.506+0000 7fc5cd400640 1 --2- 172.21.15.131:0/76363646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c80b5ef0 0x7fc5c8072370 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80b0740 tx=0x7fc5c002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fc26184c640369 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:42.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.506+0000 7fc5c7400640 1 -- 172.21.15.131:0/76363646 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0007740 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.506+0000 7fc5c7400640 1 -- 172.21.15.131:0/76363646 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c00078e0 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.507+0000 7fc5c7400640 1 -- 172.21.15.131:0/76363646 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0007cb0 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.507+0000 7fc5cf400640 1 -- 172.21.15.131:0/76363646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c80b5ef0 msgr2=0x7fc5c8072370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.507+0000 7fc5cf400640 1 --2- 172.21.15.131:0/76363646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c80b5ef0 0x7fc5c8072370 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80b0740 tx=0x7fc5c002a680 comp rx=0 tx=0).stop 2024-04-28T12:38:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.507+0000 7fc5cf400640 1 -- 172.21.15.131:0/76363646 shutdown_connections 2024-04-28T12:38:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.507+0000 7fc5cf400640 1 --2- 172.21.15.131:0/76363646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c80b5ef0 0x7fc5c8072370 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.507+0000 7fc5cf400640 1 --2- 172.21.15.131:0/76363646 >> v2:172.21.15.157:3300/0 conn(0x7fc5c815f1e0 0x7fc5c806e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.507+0000 7fc5cf400640 1 -- 172.21.15.131:0/76363646 >> 172.21.15.131:0/76363646 conn(0x7fc5c80b2900 msgr2=0x7fc5c80b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:42.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.507+0000 7fc5cf400640 1 -- 172.21.15.131:0/76363646 shutdown_connections 2024-04-28T12:38:42.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.507+0000 7fc5cf400640 1 -- 172.21.15.131:0/76363646 wait complete. 2024-04-28T12:38:42.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.509+0000 7fc5cf400640 1 Processor -- start 2024-04-28T12:38:42.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.509+0000 7fc5cf400640 1 -- start start 2024-04-28T12:38:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.510+0000 7fc5cf400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c8071ee0 0x7fc5c814a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.510+0000 7fc5cf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c80b5ef0 0x7fc5c815fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.510+0000 7fc5cf400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5c815f1e0 0x7fc5c8161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.510+0000 7fc5cf400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c815f9d0 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.510+0000 7fc5cf400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c80b42c0 con 0x7fc5c8071ee0 2024-04-28T12:38:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.510+0000 7fc5cf400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c80b3f70 con 0x7fc5c815f1e0 2024-04-28T12:38:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.510+0000 7fc5c7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c80b5ef0 0x7fc5c815fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.510+0000 7fc5c7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c80b5ef0 0x7fc5c815fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42178/0 (socket says 172.21.15.131:42178) 2024-04-28T12:38:42.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.510+0000 7fc5c7e00640 1 -- 172.21.15.131:0/1683120887 learned_addr learned my addr 172.21.15.131:0/1683120887 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:42.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5cca00640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c8071ee0 0x7fc5c814a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5cd400640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5c815f1e0 0x7fc5c8161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5c7e00640 1 -- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5c815f1e0 msgr2=0x7fc5c8161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5c7e00640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5c815f1e0 0x7fc5c8161610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5c7e00640 1 -- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c8071ee0 msgr2=0x7fc5c814a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5c7e00640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c8071ee0 0x7fc5c814a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5c7e00640 1 -- 172.21.15.131:0/1683120887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c814ac70 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5cca00640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c8071ee0 0x7fc5c814a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5c7e00640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c80b5ef0 0x7fc5c815fd20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400c430 tx=0x7fc5b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5c4c00640 1 -- 172.21.15.131:0/1683120887 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b40074e0 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5c4c00640 1 -- 172.21.15.131:0/1683120887 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b40047e0 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5cf400640 1 -- 172.21.15.131:0/1683120887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c8160570 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5c4c00640 1 -- 172.21.15.131:0/1683120887 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4004bd0 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.511+0000 7fc5cf400640 1 -- 172.21.15.131:0/1683120887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c8160b30 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.513+0000 7fc5cf400640 1 -- 172.21.15.131:0/1683120887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c8069350 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.514+0000 7fc5c4c00640 1 -- 172.21.15.131:0/1683120887 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5b4011070 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.515+0000 7fc5c4c00640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc5a806c8f0 0x7fc5a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.515+0000 7fc5c4c00640 1 -- 172.21.15.131:0/1683120887 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc5b408bc40 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.516+0000 7fc5cca00640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc5a806c8f0 0x7fc5a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.517+0000 7fc5cca00640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc5a806c8f0 0x7fc5a806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc006fd0 tx=0x7fc5bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:42.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.520+0000 7fc5c4c00640 1 -- 172.21.15.131:0/1683120887 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc5c814ac70 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.715+0000 7fc5cf400640 1 -- 172.21.15.131:0/1683120887 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc5c806c780 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.716+0000 7fc5c4c00640 1 -- 172.21.15.131:0/1683120887 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fc5b4059bd0 con 0x7fc5c80b5ef0 2024-04-28T12:38:42.712 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:42.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.718+0000 7fc5cf400640 1 -- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc5a806c8f0 msgr2=0x7fc5a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.718+0000 7fc5cf400640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc5a806c8f0 0x7fc5a806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc006fd0 tx=0x7fc5bc008040 comp rx=0 tx=0).stop 2024-04-28T12:38:42.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.718+0000 7fc5cf400640 1 -- 172.21.15.131:0/1683120887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c80b5ef0 msgr2=0x7fc5c815fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.718+0000 7fc5cf400640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c80b5ef0 0x7fc5c815fd20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400c430 tx=0x7fc5b400c8f0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.718+0000 7fc5cf400640 1 -- 172.21.15.131:0/1683120887 shutdown_connections 2024-04-28T12:38:42.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.718+0000 7fc5cf400640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc5a806c8f0 0x7fc5a806edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.718+0000 7fc5cf400640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5c815f1e0 0x7fc5c8161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.718+0000 7fc5cf400640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c80b5ef0 0x7fc5c815fd20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.718+0000 7fc5cf400640 1 --2- 172.21.15.131:0/1683120887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5c8071ee0 0x7fc5c814a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.718+0000 7fc5cf400640 1 -- 172.21.15.131:0/1683120887 >> 172.21.15.131:0/1683120887 conn(0x7fc5c80b2900 msgr2=0x7fc5c80af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:42.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.718+0000 7fc5cf400640 1 -- 172.21.15.131:0/1683120887 shutdown_connections 2024-04-28T12:38:42.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.718+0000 7fc5cf400640 1 -- 172.21.15.131:0/1683120887 wait complete. 2024-04-28T12:38:42.732 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:42.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.941+0000 7fac2be00640 1 Processor -- start 2024-04-28T12:38:42.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.941+0000 7fac2be00640 1 -- start start 2024-04-28T12:38:42.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.943+0000 7fac2be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac2c0affb0 0x7fac2c0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.943+0000 7fac2be00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac2c15d2e0 con 0x7fac2c1572b0 2024-04-28T12:38:42.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.943+0000 7fac2be00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac2c15d5c0 con 0x7fac2c0d12e0 2024-04-28T12:38:42.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.943+0000 7fac2be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac2c15d8a0 con 0x7fac2c0affb0 2024-04-28T12:38:42.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.943+0000 7fac2aa00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fac2c1572b0 0x7fac2c0d0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49320/0 (socket says 172.21.15.131:49320) 2024-04-28T12:38:42.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.943+0000 7fac2aa00640 1 -- 172.21.15.131:0/1682137755 learned_addr learned my addr 172.21.15.131:0/1682137755 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:42.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.943+0000 7fac2b400640 1 --2- 172.21.15.131:0/1682137755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac2c0affb0 0x7fac2c0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.944+0000 7fac29600640 1 -- 172.21.15.131:0/1682137755 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1177615398 0 0) 0x7fac2c15d2e0 con 0x7fac2c1572b0 2024-04-28T12:38:42.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.944+0000 7fac29600640 1 -- 172.21.15.131:0/1682137755 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac10003520 con 0x7fac2c1572b0 2024-04-28T12:38:42.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.944+0000 7fac2b400640 1 -- 172.21.15.131:0/1682137755 >> v1:172.21.15.157:6789/0 conn(0x7fac2c0d12e0 legacy=0x7fac2c0af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.944+0000 7fac2b400640 1 -- 172.21.15.131:0/1682137755 >> v1:172.21.15.131:6789/0 conn(0x7fac2c1572b0 legacy=0x7fac2c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.944+0000 7fac2b400640 1 -- 172.21.15.131:0/1682137755 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c15db80 con 0x7fac2c0affb0 2024-04-28T12:38:42.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.946+0000 7fac2b400640 1 --2- 172.21.15.131:0/1682137755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac2c0affb0 0x7fac2c0b3e30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac24009dc0 tx=0x7fac2402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a999f120d0405e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:42.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.946+0000 7fac29600640 1 -- 172.21.15.131:0/1682137755 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2402ae60 con 0x7fac2c0affb0 2024-04-28T12:38:42.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.946+0000 7fac29600640 1 -- 172.21.15.131:0/1682137755 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac24007740 con 0x7fac2c0affb0 2024-04-28T12:38:42.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.946+0000 7fac29600640 1 -- 172.21.15.131:0/1682137755 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24007af0 con 0x7fac2c0affb0 2024-04-28T12:38:42.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.947+0000 7fac2be00640 1 -- 172.21.15.131:0/1682137755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac2c0affb0 msgr2=0x7fac2c0b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.947+0000 7fac2be00640 1 --2- 172.21.15.131:0/1682137755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac2c0affb0 0x7fac2c0b3e30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac24009dc0 tx=0x7fac2402a640 comp rx=0 tx=0).stop 2024-04-28T12:38:42.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.947+0000 7fac2be00640 1 -- 172.21.15.131:0/1682137755 shutdown_connections 2024-04-28T12:38:42.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.947+0000 7fac2be00640 1 --2- 172.21.15.131:0/1682137755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac2c0affb0 0x7fac2c0b3e30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.947+0000 7fac2be00640 1 -- 172.21.15.131:0/1682137755 >> 172.21.15.131:0/1682137755 conn(0x7fac2c0684e0 msgr2=0x7fac2c0688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:42.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.947+0000 7fac2be00640 1 -- 172.21.15.131:0/1682137755 shutdown_connections 2024-04-28T12:38:42.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.947+0000 7fac2be00640 1 -- 172.21.15.131:0/1682137755 wait complete. 2024-04-28T12:38:42.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.949+0000 7fac2be00640 1 Processor -- start 2024-04-28T12:38:42.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.950+0000 7fac2be00640 1 -- start start 2024-04-28T12:38:42.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac2c0affb0 0x7fac2c1481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac2c0d12e0 0x7fac2c148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac2c1572b0 0x7fac2c0ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2be00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c160c80 con 0x7fac2c0affb0 2024-04-28T12:38:42.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c160b00 con 0x7fac2c1572b0 2024-04-28T12:38:42.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac2c160e00 con 0x7fac2c0d12e0 2024-04-28T12:38:42.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac2c0affb0 0x7fac2c1481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac2c0affb0 0x7fac2c1481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42192/0 (socket says 172.21.15.131:42192) 2024-04-28T12:38:42.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2aa00640 1 -- 172.21.15.131:0/3409344072 learned_addr learned my addr 172.21.15.131:0/3409344072 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:42.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2aa00640 1 -- 172.21.15.131:0/3409344072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac2c0d12e0 msgr2=0x7fac2c148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2aa00640 1 --2- 172.21.15.131:0/3409344072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac2c0d12e0 0x7fac2c148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2aa00640 1 -- 172.21.15.131:0/3409344072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac2c1572b0 msgr2=0x7fac2c0ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:42.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2aa00640 1 --2- 172.21.15.131:0/3409344072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac2c1572b0 0x7fac2c0ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:42.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.951+0000 7fac2aa00640 1 -- 172.21.15.131:0/3409344072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c157a70 con 0x7fac2c0affb0 2024-04-28T12:38:42.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.952+0000 7fac2aa00640 1 --2- 172.21.15.131:0/3409344072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac2c0affb0 0x7fac2c1481e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00c440 tx=0x7fac1c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:42.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.952+0000 7fac22a00640 1 -- 172.21.15.131:0/3409344072 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c017070 con 0x7fac2c0affb0 2024-04-28T12:38:42.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.952+0000 7fac22a00640 1 -- 172.21.15.131:0/3409344072 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac1c005ce0 con 0x7fac2c0affb0 2024-04-28T12:38:42.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.952+0000 7fac22a00640 1 -- 172.21.15.131:0/3409344072 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0072b0 con 0x7fac2c0affb0 2024-04-28T12:38:42.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.952+0000 7fac2be00640 1 -- 172.21.15.131:0/3409344072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c157d50 con 0x7fac2c0affb0 2024-04-28T12:38:42.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.952+0000 7fac2be00640 1 -- 172.21.15.131:0/3409344072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c158310 con 0x7fac2c0affb0 2024-04-28T12:38:42.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.954+0000 7fac2be00640 1 -- 172.21.15.131:0/3409344072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2c0506c0 con 0x7fac2c0affb0 2024-04-28T12:38:42.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.954+0000 7fac22a00640 1 -- 172.21.15.131:0/3409344072 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac1c01a070 con 0x7fac2c0affb0 2024-04-28T12:38:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.955+0000 7fac22a00640 1 --2- 172.21.15.131:0/3409344072 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac0c06c8f0 0x7fac0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.955+0000 7fac22a00640 1 -- 172.21.15.131:0/3409344072 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fac1c08b3d0 con 0x7fac2c0affb0 2024-04-28T12:38:42.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.956+0000 7fac2a000640 1 --2- 172.21.15.131:0/3409344072 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac0c06c8f0 0x7fac0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:42.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.957+0000 7fac2a000640 1 --2- 172.21.15.131:0/3409344072 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac0c06c8f0 0x7fac0c06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac18000f60 tx=0x7fac180079d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:42.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:42.961+0000 7fac22a00640 1 -- 172.21.15.131:0/3409344072 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fac1c059420 con 0x7fac2c0affb0 2024-04-28T12:38:43.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.161+0000 7fac2be00640 1 -- 172.21.15.131:0/3409344072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fac2c0d0df0 con 0x7fac2c0affb0 2024-04-28T12:38:43.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.162+0000 7fac22a00640 1 -- 172.21.15.131:0/3409344072 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fac1c058d20 con 0x7fac2c0affb0 2024-04-28T12:38:43.158 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:43.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.164+0000 7fac2be00640 1 -- 172.21.15.131:0/3409344072 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac0c06c8f0 msgr2=0x7fac0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:43.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.164+0000 7fac2be00640 1 --2- 172.21.15.131:0/3409344072 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac0c06c8f0 0x7fac0c06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac18000f60 tx=0x7fac180079d0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.164+0000 7fac2be00640 1 -- 172.21.15.131:0/3409344072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac2c0affb0 msgr2=0x7fac2c1481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:43.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.164+0000 7fac2be00640 1 --2- 172.21.15.131:0/3409344072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac2c0affb0 0x7fac2c1481e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00c440 tx=0x7fac1c00c900 comp rx=0 tx=0).stop 2024-04-28T12:38:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.164+0000 7fac2be00640 1 -- 172.21.15.131:0/3409344072 shutdown_connections 2024-04-28T12:38:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.164+0000 7fac2be00640 1 --2- 172.21.15.131:0/3409344072 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac0c06c8f0 0x7fac0c06edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.164+0000 7fac2be00640 1 --2- 172.21.15.131:0/3409344072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac2c1572b0 0x7fac2c0ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.164+0000 7fac2be00640 1 --2- 172.21.15.131:0/3409344072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac2c0d12e0 0x7fac2c148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.164+0000 7fac2be00640 1 --2- 172.21.15.131:0/3409344072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac2c0affb0 0x7fac2c1481e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.164+0000 7fac2be00640 1 -- 172.21.15.131:0/3409344072 >> 172.21.15.131:0/3409344072 conn(0x7fac2c0684e0 msgr2=0x7fac2c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.164+0000 7fac2be00640 1 -- 172.21.15.131:0/3409344072 shutdown_connections 2024-04-28T12:38:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.165+0000 7fac2be00640 1 -- 172.21.15.131:0/3409344072 wait complete. 2024-04-28T12:38:43.181 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:43.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.388+0000 7fdd0b200640 1 Processor -- start 2024-04-28T12:38:43.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.388+0000 7fdd0b200640 1 -- start start 2024-04-28T12:38:43.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.390+0000 7fdd0b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c0d3830 0x7fdd0c0d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.390+0000 7fdd0b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdd0c0b3cb0 0x7fdd0c0b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.390+0000 7fdd0b200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd0c0d2d70 con 0x7fdd0c15f180 2024-04-28T12:38:43.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.390+0000 7fdd0b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fdd0c15f560 con 0x7fdd0c0b3cb0 2024-04-28T12:38:43.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.390+0000 7fdd0b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0c0d06e0 con 0x7fdd0c0d3830 2024-04-28T12:38:43.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.390+0000 7fdd09e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdd0c15f180 0x7fdd0c0d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49330/0 (socket says 172.21.15.131:49330) 2024-04-28T12:38:43.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.390+0000 7fdd09e00640 1 -- 172.21.15.131:0/3101119454 learned_addr learned my addr 172.21.15.131:0/3101119454 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:43.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.390+0000 7fdd0a800640 1 --2- 172.21.15.131:0/3101119454 >> v2:172.21.15.131:3300/0 conn(0x7fdd0c0b3cb0 0x7fdd0c0b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:43.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.391+0000 7fdd09400640 1 --2- 172.21.15.131:0/3101119454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c0d3830 0x7fdd0c0d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:43.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.391+0000 7fdd08a00640 1 -- 172.21.15.131:0/3101119454 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3068158013 0 0) 0x7fdd0c0d2d70 con 0x7fdd0c15f180 2024-04-28T12:38:43.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.391+0000 7fdd08a00640 1 -- 172.21.15.131:0/3101119454 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcf0003520 con 0x7fdd0c15f180 2024-04-28T12:38:43.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.391+0000 7fdd0a800640 1 -- 172.21.15.131:0/3101119454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c0d3830 msgr2=0x7fdd0c0d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:43.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.391+0000 7fdd0a800640 1 --2- 172.21.15.131:0/3101119454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c0d3830 0x7fdd0c0d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.391+0000 7fdd0a800640 1 -- 172.21.15.131:0/3101119454 >> v1:172.21.15.131:6789/0 conn(0x7fdd0c15f180 legacy=0x7fdd0c0d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:43.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.391+0000 7fdd0a800640 1 -- 172.21.15.131:0/3101119454 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c0d09c0 con 0x7fdd0c0b3cb0 2024-04-28T12:38:43.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.391+0000 7fdd09400640 1 --2- 172.21.15.131:0/3101119454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c0d3830 0x7fdd0c0d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:43.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.392+0000 7fdd0a800640 1 --2- 172.21.15.131:0/3101119454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd0c0b3cb0 0x7fdd0c0b1740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdd040097a0 tx=0x7fdd0402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8d9b9aed92bd80 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:43.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.392+0000 7fdd08a00640 1 -- 172.21.15.131:0/3101119454 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04037070 con 0x7fdd0c0b3cb0 2024-04-28T12:38:43.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.392+0000 7fdd08a00640 1 -- 172.21.15.131:0/3101119454 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd04032440 con 0x7fdd0c0b3cb0 2024-04-28T12:38:43.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.392+0000 7fdd08a00640 1 -- 172.21.15.131:0/3101119454 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04032810 con 0x7fdd0c0b3cb0 2024-04-28T12:38:43.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.393+0000 7fdd0b200640 1 -- 172.21.15.131:0/3101119454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd0c0b3cb0 msgr2=0x7fdd0c0b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:43.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.393+0000 7fdd0b200640 1 --2- 172.21.15.131:0/3101119454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd0c0b3cb0 0x7fdd0c0b1740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdd040097a0 tx=0x7fdd0402a180 comp rx=0 tx=0).stop 2024-04-28T12:38:43.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.393+0000 7fdd0b200640 1 -- 172.21.15.131:0/3101119454 shutdown_connections 2024-04-28T12:38:43.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.393+0000 7fdd0b200640 1 --2- 172.21.15.131:0/3101119454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd0c0b3cb0 0x7fdd0c0b1740 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.393+0000 7fdd0b200640 1 --2- 172.21.15.131:0/3101119454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c0d3830 0x7fdd0c0d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.393+0000 7fdd0b200640 1 -- 172.21.15.131:0/3101119454 >> 172.21.15.131:0/3101119454 conn(0x7fdd0c0d4bd0 msgr2=0x7fdd0c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:43.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.393+0000 7fdd0b200640 1 -- 172.21.15.131:0/3101119454 shutdown_connections 2024-04-28T12:38:43.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.393+0000 7fdd0b200640 1 -- 172.21.15.131:0/3101119454 wait complete. 2024-04-28T12:38:43.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.395+0000 7fdd0b200640 1 Processor -- start 2024-04-28T12:38:43.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.395+0000 7fdd0b200640 1 -- start start 2024-04-28T12:38:43.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.396+0000 7fdd0b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0c0b3cb0 0x7fdd0c1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.396+0000 7fdd0b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd0c0d3830 0x7fdd0c0ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.396+0000 7fdd0b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c15f180 0x7fdd0c0ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.396+0000 7fdd0b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c0d11c0 con 0x7fdd0c0d3830 2024-04-28T12:38:43.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.396+0000 7fdd0b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c0d2a40 con 0x7fdd0c0b3cb0 2024-04-28T12:38:43.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.396+0000 7fdd0b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0c0cd680 con 0x7fdd0c15f180 2024-04-28T12:38:43.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.396+0000 7fdd09400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd0c0d3830 0x7fdd0c0ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:43.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.396+0000 7fdd09400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd0c0d3830 0x7fdd0c0ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42206/0 (socket says 172.21.15.131:42206) 2024-04-28T12:38:43.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.396+0000 7fdd09400640 1 -- 172.21.15.131:0/295690587 learned_addr learned my addr 172.21.15.131:0/295690587 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:43.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.397+0000 7fdd09400640 1 -- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c15f180 msgr2=0x7fdd0c0ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:43.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.397+0000 7fdd09400640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c15f180 0x7fdd0c0ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.397+0000 7fdd09e00640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0c0b3cb0 0x7fdd0c1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:43.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.397+0000 7fdd09400640 1 -- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0c0b3cb0 msgr2=0x7fdd0c1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:43.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.397+0000 7fdd09400640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0c0b3cb0 0x7fdd0c1594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.397+0000 7fdd09400640 1 -- 172.21.15.131:0/295690587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c158720 con 0x7fdd0c0d3830 2024-04-28T12:38:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.397+0000 7fdd09400640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd0c0d3830 0x7fdd0c0ca3e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c7f0 tx=0x7fdcf800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.398+0000 7fdd02000640 1 -- 172.21.15.131:0/295690587 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8015070 con 0x7fdd0c0d3830 2024-04-28T12:38:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.398+0000 7fdd02000640 1 -- 172.21.15.131:0/295690587 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf8007a30 con 0x7fdd0c0d3830 2024-04-28T12:38:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.398+0000 7fdd02000640 1 -- 172.21.15.131:0/295690587 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8007e00 con 0x7fdd0c0d3830 2024-04-28T12:38:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.398+0000 7fdd0b200640 1 -- 172.21.15.131:0/295690587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c158a00 con 0x7fdd0c0d3830 2024-04-28T12:38:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.399+0000 7fdd0b200640 1 -- 172.21.15.131:0/295690587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c159a20 con 0x7fdd0c0d3830 2024-04-28T12:38:43.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.400+0000 7fdd0b200640 1 -- 172.21.15.131:0/295690587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0c0506c0 con 0x7fdd0c0d3830 2024-04-28T12:38:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.401+0000 7fdd02000640 1 -- 172.21.15.131:0/295690587 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcf8018070 con 0x7fdd0c0d3830 2024-04-28T12:38:43.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.401+0000 7fdd02000640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdcec06c940 0x7fdcec06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.402+0000 7fdd02000640 1 -- 172.21.15.131:0/295690587 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdcf808beb0 con 0x7fdd0c0d3830 2024-04-28T12:38:43.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.402+0000 7fdd09e00640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdcec06c940 0x7fdcec06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:43.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.403+0000 7fdd09e00640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdcec06c940 0x7fdcec06ee00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0012e0 tx=0x7fdcfc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:43.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.406+0000 7fdd02000640 1 -- 172.21.15.131:0/295690587 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdcf8059f50 con 0x7fdd0c0d3830 2024-04-28T12:38:43.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.606+0000 7fdd0b200640 1 -- 172.21.15.131:0/295690587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdd0c15e3a0 con 0x7fdd0c0d3830 2024-04-28T12:38:43.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.607+0000 7fdd02000640 1 -- 172.21.15.131:0/295690587 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fdcf8059850 con 0x7fdd0c0d3830 2024-04-28T12:38:43.604 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.610+0000 7fdd0b200640 1 -- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdcec06c940 msgr2=0x7fdcec06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.610+0000 7fdd0b200640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdcec06c940 0x7fdcec06ee00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0012e0 tx=0x7fdcfc009040 comp rx=0 tx=0).stop 2024-04-28T12:38:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.610+0000 7fdd0b200640 1 -- 172.21.15.131:0/295690587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd0c0d3830 msgr2=0x7fdd0c0ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.610+0000 7fdd0b200640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd0c0d3830 0x7fdd0c0ca3e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c7f0 tx=0x7fdcf800cbf0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.610+0000 7fdd0b200640 1 -- 172.21.15.131:0/295690587 shutdown_connections 2024-04-28T12:38:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.610+0000 7fdd0b200640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdcec06c940 0x7fdcec06ee00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.610+0000 7fdd0b200640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c15f180 0x7fdd0c0ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.610+0000 7fdd0b200640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd0c0d3830 0x7fdd0c0ca3e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.610+0000 7fdd0b200640 1 --2- 172.21.15.131:0/295690587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0c0b3cb0 0x7fdd0c1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.610+0000 7fdd0b200640 1 -- 172.21.15.131:0/295690587 >> 172.21.15.131:0/295690587 conn(0x7fdd0c0d4bd0 msgr2=0x7fdd0c15d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.610+0000 7fdd0b200640 1 -- 172.21.15.131:0/295690587 shutdown_connections 2024-04-28T12:38:43.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.610+0000 7fdd0b200640 1 -- 172.21.15.131:0/295690587 wait complete. 2024-04-28T12:38:43.629 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T12:38:43.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.816+0000 7fcb15400640 1 Processor -- start 2024-04-28T12:38:43.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.816+0000 7fcb15400640 1 -- start start 2024-04-28T12:38:43.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.818+0000 7fcb15400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcb1015f170 0x7fcb100d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.818+0000 7fcb15400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fcb100d5850 0x7fcb100d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.818+0000 7fcb15400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb100d36a0 0x7fcb100cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.818+0000 7fcb15400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcb100d0f10 con 0x7fcb1015f170 2024-04-28T12:38:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.818+0000 7fcb15400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fcb100d0c30 con 0x7fcb100d5850 2024-04-28T12:38:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.818+0000 7fcb15400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcb100b42a0 con 0x7fcb100d36a0 2024-04-28T12:38:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.818+0000 7fcb0fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcb1015f170 0x7fcb100d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.818+0000 7fcb0fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcb1015f170 0x7fcb100d2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42220/0 (socket says 172.21.15.131:42220) 2024-04-28T12:38:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.818+0000 7fcb0fe00640 1 -- 172.21.15.131:0/2409913792 learned_addr learned my addr 172.21.15.131:0/2409913792 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.819+0000 7fcb14a00640 1 --2- 172.21.15.131:0/2409913792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb100d36a0 0x7fcb100cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.819+0000 7fcb0f400640 1 --2- 172.21.15.131:0/2409913792 >> v2:172.21.15.157:3300/0 conn(0x7fcb100d5850 0x7fcb100d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.819+0000 7fcb0fe00640 1 -- 172.21.15.131:0/2409913792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb100d36a0 msgr2=0x7fcb100cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.819+0000 7fcb0fe00640 1 --2- 172.21.15.131:0/2409913792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb100d36a0 0x7fcb100cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.819+0000 7fcb0fe00640 1 -- 172.21.15.131:0/2409913792 >> v2:172.21.15.157:3300/0 conn(0x7fcb100d5850 msgr2=0x7fcb100d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.819+0000 7fcb0fe00640 1 --2- 172.21.15.131:0/2409913792 >> v2:172.21.15.157:3300/0 conn(0x7fcb100d5850 0x7fcb100d3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.819+0000 7fcb0fe00640 1 -- 172.21.15.131:0/2409913792 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb100b4580 con 0x7fcb1015f170 2024-04-28T12:38:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.819+0000 7fcb0f400640 1 --2- 172.21.15.131:0/2409913792 >> v2:172.21.15.157:3300/0 conn(0x7fcb100d5850 0x7fcb100d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.820+0000 7fcb0fe00640 1 --2- 172.21.15.131:0/2409913792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb1015f170 0x7fcb100d2a80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcb1008d2e0 tx=0x7fcb0002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa680c24004caa64 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.820+0000 7fcb0ea00640 1 -- 172.21.15.131:0/2409913792 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00037070 con 0x7fcb1015f170 2024-04-28T12:38:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.820+0000 7fcb0ea00640 1 -- 172.21.15.131:0/2409913792 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb00032440 con 0x7fcb1015f170 2024-04-28T12:38:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.820+0000 7fcb0ea00640 1 -- 172.21.15.131:0/2409913792 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00032810 con 0x7fcb1015f170 2024-04-28T12:38:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.821+0000 7fcb15400640 1 -- 172.21.15.131:0/2409913792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb1015f170 msgr2=0x7fcb100d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.821+0000 7fcb15400640 1 --2- 172.21.15.131:0/2409913792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb1015f170 0x7fcb100d2a80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcb1008d2e0 tx=0x7fcb0002a1e0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.821+0000 7fcb15400640 1 -- 172.21.15.131:0/2409913792 shutdown_connections 2024-04-28T12:38:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.821+0000 7fcb15400640 1 --2- 172.21.15.131:0/2409913792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb100d36a0 0x7fcb100cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.821+0000 7fcb15400640 1 --2- 172.21.15.131:0/2409913792 >> v2:172.21.15.157:3300/0 conn(0x7fcb100d5850 0x7fcb100d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.821+0000 7fcb15400640 1 --2- 172.21.15.131:0/2409913792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb1015f170 0x7fcb100d2a80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.821+0000 7fcb15400640 1 -- 172.21.15.131:0/2409913792 >> 172.21.15.131:0/2409913792 conn(0x7fcb100d4ca0 msgr2=0x7fcb100d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:43.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.821+0000 7fcb15400640 1 -- 172.21.15.131:0/2409913792 shutdown_connections 2024-04-28T12:38:43.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.821+0000 7fcb15400640 1 -- 172.21.15.131:0/2409913792 wait complete. 2024-04-28T12:38:43.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.823+0000 7fcb15400640 1 Processor -- start 2024-04-28T12:38:43.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.823+0000 7fcb15400640 1 -- start start 2024-04-28T12:38:43.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.824+0000 7fcb15400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb100d36a0 0x7fcb1014e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.824+0000 7fcb15400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb100d5850 0x7fcb100b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.824+0000 7fcb15400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb1015f170 0x7fcb100b9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.824+0000 7fcb15400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb100cfd80 con 0x7fcb100d5850 2024-04-28T12:38:43.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.824+0000 7fcb15400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1015f550 con 0x7fcb100d36a0 2024-04-28T12:38:43.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.824+0000 7fcb15400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcb100cff00 con 0x7fcb1015f170 2024-04-28T12:38:43.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.824+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb100d5850 0x7fcb100b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:43.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.824+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb100d5850 0x7fcb100b77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42224/0 (socket says 172.21.15.131:42224) 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.824+0000 7fcb0f400640 1 -- 172.21.15.131:0/4245955360 learned_addr learned my addr 172.21.15.131:0/4245955360 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.825+0000 7fcb0fe00640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb100d36a0 0x7fcb1014e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.825+0000 7fcb14a00640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb1015f170 0x7fcb100b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.825+0000 7fcb0f400640 1 -- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb1015f170 msgr2=0x7fcb100b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.825+0000 7fcb0f400640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb1015f170 0x7fcb100b9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.825+0000 7fcb0f400640 1 -- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb100d36a0 msgr2=0x7fcb1014e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.825+0000 7fcb0f400640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb100d36a0 0x7fcb1014e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.825+0000 7fcb0f400640 1 -- 172.21.15.131:0/4245955360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1014e610 con 0x7fcb100d5850 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.825+0000 7fcb0f400640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb100d5850 0x7fcb100b77f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00cf40 tx=0x7fcafc00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.825+0000 7fcb07e00640 1 -- 172.21.15.131:0/4245955360 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc01a070 con 0x7fcb100d5850 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.825+0000 7fcb07e00640 1 -- 172.21.15.131:0/4245955360 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcafc00d840 con 0x7fcb100d5850 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.825+0000 7fcb07e00640 1 -- 172.21.15.131:0/4245955360 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00dc30 con 0x7fcb100d5850 2024-04-28T12:38:43.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.825+0000 7fcb15400640 1 -- 172.21.15.131:0/4245955360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb100b8160 con 0x7fcb100d5850 2024-04-28T12:38:43.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.826+0000 7fcb15400640 1 -- 172.21.15.131:0/4245955360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb100b8720 con 0x7fcb100d5850 2024-04-28T12:38:43.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.827+0000 7fcb15400640 1 -- 172.21.15.131:0/4245955360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb10069a60 con 0x7fcb100d5850 2024-04-28T12:38:43.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.828+0000 7fcb07e00640 1 -- 172.21.15.131:0/4245955360 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcafc01d070 con 0x7fcb100d5850 2024-04-28T12:38:43.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.828+0000 7fcb07e00640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcaf006c8f0 0x7fcaf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:43.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.829+0000 7fcb07e00640 1 -- 172.21.15.131:0/4245955360 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcafc08c6f0 con 0x7fcb100d5850 2024-04-28T12:38:43.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.829+0000 7fcb0fe00640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcaf006c8f0 0x7fcaf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:43.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.830+0000 7fcb0fe00640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcaf006c8f0 0x7fcaf006edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb00034000 tx=0x7fcb0002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:43.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:43.833+0000 7fcb07e00640 1 -- 172.21.15.131:0/4245955360 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb1014e610 con 0x7fcb100d5850 2024-04-28T12:38:44.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.045+0000 7fcb15400640 1 -- 172.21.15.131:0/4245955360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcb100d0310 con 0x7fcb100d5850 2024-04-28T12:38:44.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.046+0000 7fcb07e00640 1 -- 172.21.15.131:0/4245955360 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fcafc05a030 con 0x7fcb100d5850 2024-04-28T12:38:44.042 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-04-28T12:38:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.049+0000 7fcb15400640 1 -- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcaf006c8f0 msgr2=0x7fcaf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.049+0000 7fcb15400640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcaf006c8f0 0x7fcaf006edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb00034000 tx=0x7fcb0002e000 comp rx=0 tx=0).stop 2024-04-28T12:38:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.049+0000 7fcb15400640 1 -- 172.21.15.131:0/4245955360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb100d5850 msgr2=0x7fcb100b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.049+0000 7fcb15400640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb100d5850 0x7fcb100b77f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00cf40 tx=0x7fcafc00a7d0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.049+0000 7fcb15400640 1 -- 172.21.15.131:0/4245955360 shutdown_connections 2024-04-28T12:38:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.049+0000 7fcb15400640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcaf006c8f0 0x7fcaf006edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.049+0000 7fcb15400640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb1015f170 0x7fcb100b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.049+0000 7fcb15400640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb100d5850 0x7fcb100b77f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.049+0000 7fcb15400640 1 --2- 172.21.15.131:0/4245955360 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb100d36a0 0x7fcb1014e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.049+0000 7fcb15400640 1 -- 172.21.15.131:0/4245955360 >> 172.21.15.131:0/4245955360 conn(0x7fcb100d4ca0 msgr2=0x7fcb100d3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.049+0000 7fcb15400640 1 -- 172.21.15.131:0/4245955360 shutdown_connections 2024-04-28T12:38:44.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.049+0000 7fcb15400640 1 -- 172.21.15.131:0/4245955360 wait complete. 2024-04-28T12:38:44.063 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:38:44.064 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:38:44.115 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:44.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.317+0000 7fc301000640 1 Processor -- start 2024-04-28T12:38:44.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.317+0000 7fc301000640 1 -- start start 2024-04-28T12:38:44.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.318+0000 7fc301000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15cab0 0x7fc2fc0afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.319+0000 7fc301000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc2fc0d1250 0x7fc2fc15f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.319+0000 7fc301000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2fc0b0650 con 0x7fc2fc0b0020 2024-04-28T12:38:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.319+0000 7fc301000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc2fc0d0900 con 0x7fc2fc0d1250 2024-04-28T12:38:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.319+0000 7fc301000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc2fc1635a0 con 0x7fc2fc15cab0 2024-04-28T12:38:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.319+0000 7fc2f9400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc2fc0b0020 0x7fc2fc0d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49344/0 (socket says 172.21.15.131:49344) 2024-04-28T12:38:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.319+0000 7fc2f9400640 1 -- 172.21.15.131:0/287650577 learned_addr learned my addr 172.21.15.131:0/287650577 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.319+0000 7fc2f9e00640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15cab0 0x7fc2fc0afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.319+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2584563601 0 0) 0x7fc2fc0b0650 con 0x7fc2fc0b0020 2024-04-28T12:38:44.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2e0003520 con 0x7fc2fc0b0020 2024-04-28T12:38:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2fa800640 1 --2- 172.21.15.131:0/287650577 >> v2:172.21.15.157:3300/0 conn(0x7fc2fc0d1250 0x7fc2fc15f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3171199477 0 0) 0x7fc2e0003520 con 0x7fc2fc0b0020 2024-04-28T12:38:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15cab0 msgr2=0x7fc2fc0afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15cab0 0x7fc2fc0afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 >> v2:172.21.15.157:3300/0 conn(0x7fc2fc0d1250 msgr2=0x7fc2fc15f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 --2- 172.21.15.131:0/287650577 >> v2:172.21.15.157:3300/0 conn(0x7fc2fc0d1250 0x7fc2fc15f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc163a30 con 0x7fc2fc0b0020 2024-04-28T12:38:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fc2e8001670 con 0x7fc2fc0b0020 2024-04-28T12:38:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 >> v1:172.21.15.131:6789/0 conn(0x7fc2fc0b0020 legacy=0x7fc2fc0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2e00039b0 0x7fc2e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15cab0 0x7fc2e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e0004d40 0x7fc2e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc1635a0 con 0x7fc2e00039b0 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e000b9c0 con 0x7fc2e0004d40 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e000bbc0 con 0x7fc2fc15cab0 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.320+0000 7fc2f9e00640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2e00039b0 0x7fc2e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.321+0000 7fc2f9400640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15cab0 0x7fc2e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.321+0000 7fc2f9e00640 1 -- 172.21.15.131:0/287650577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15cab0 msgr2=0x7fc2e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.321+0000 7fc2f9e00640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15cab0 0x7fc2e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.321+0000 7fc2f9e00640 1 -- 172.21.15.131:0/287650577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e0004d40 msgr2=0x7fc2e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.321+0000 7fc2fa800640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e0004d40 0x7fc2e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.321+0000 7fc2f9e00640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e0004d40 0x7fc2e00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.321+0000 7fc2f9e00640 1 -- 172.21.15.131:0/287650577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e0003520 con 0x7fc2e00039b0 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.321+0000 7fc2f9400640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15cab0 0x7fc2e0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.322+0000 7fc2f9e00640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2e00039b0 0x7fc2e0003dd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009ac0 tx=0x7fc2e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea1e3ad324f0052 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.322+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec036070 con 0x7fc2e00039b0 2024-04-28T12:38:44.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.322+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2ec031400 con 0x7fc2e00039b0 2024-04-28T12:38:44.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.322+0000 7fc2f8a00640 1 -- 172.21.15.131:0/287650577 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec009690 con 0x7fc2e00039b0 2024-04-28T12:38:44.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.323+0000 7fc301000640 1 -- 172.21.15.131:0/287650577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2e00039b0 msgr2=0x7fc2e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.323+0000 7fc301000640 1 --2- 172.21.15.131:0/287650577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2e00039b0 0x7fc2e0003dd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009ac0 tx=0x7fc2e0005830 comp rx=0 tx=0).stop 2024-04-28T12:38:44.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.323+0000 7fc2fa800640 1 -- 172.21.15.131:0/287650577 reap_dead start 2024-04-28T12:38:44.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.323+0000 7fc301000640 1 -- 172.21.15.131:0/287650577 shutdown_connections 2024-04-28T12:38:44.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.323+0000 7fc301000640 1 -- 172.21.15.131:0/287650577 >> 172.21.15.131:0/287650577 conn(0x7fc2fc0b2900 msgr2=0x7fc2fc0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:44.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.323+0000 7fc301000640 1 -- 172.21.15.131:0/287650577 shutdown_connections 2024-04-28T12:38:44.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.323+0000 7fc301000640 1 -- 172.21.15.131:0/287650577 wait complete. 2024-04-28T12:38:44.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.325+0000 7fc301000640 1 Processor -- start 2024-04-28T12:38:44.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.325+0000 7fc301000640 1 -- start start 2024-04-28T12:38:44.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc301000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e00039b0 0x7fc2fc145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc301000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15b6c0 0x7fc2fc148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc301000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2fc158770 0x7fc2fc1570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc301000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc163230 con 0x7fc2fc158770 2024-04-28T12:38:44.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc301000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc162f50 con 0x7fc2e00039b0 2024-04-28T12:38:44.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc301000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc2fc15ce90 con 0x7fc2fc15b6c0 2024-04-28T12:38:44.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc2fa800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2fc158770 0x7fc2fc1570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc2fa800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2fc158770 0x7fc2fc1570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42242/0 (socket says 172.21.15.131:42242) 2024-04-28T12:38:44.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc2fa800640 1 -- 172.21.15.131:0/816130303 learned_addr learned my addr 172.21.15.131:0/816130303 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc2f9e00640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e00039b0 0x7fc2fc145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc2f9400640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15b6c0 0x7fc2fc148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc2fa800640 1 -- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15b6c0 msgr2=0x7fc2fc148a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc2fa800640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15b6c0 0x7fc2fc148a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc2fa800640 1 -- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e00039b0 msgr2=0x7fc2fc145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc2fa800640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e00039b0 0x7fc2fc145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc2fa800640 1 -- 172.21.15.131:0/816130303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc148280 con 0x7fc2fc158770 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.326+0000 7fc2f9400640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15b6c0 0x7fc2fc148a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.327+0000 7fc2fa800640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2fc158770 0x7fc2fc1570f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009820 tx=0x7fc2f4009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.327+0000 7fc2f2000640 1 -- 172.21.15.131:0/816130303 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4019070 con 0x7fc2fc158770 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.327+0000 7fc2f2000640 1 -- 172.21.15.131:0/816130303 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f4007ce0 con 0x7fc2fc158770 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.327+0000 7fc2f2000640 1 -- 172.21.15.131:0/816130303 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4014640 con 0x7fc2fc158770 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.327+0000 7fc301000640 1 -- 172.21.15.131:0/816130303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2fc159610 con 0x7fc2fc158770 2024-04-28T12:38:44.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.327+0000 7fc301000640 1 -- 172.21.15.131:0/816130303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2fc159330 con 0x7fc2fc158770 2024-04-28T12:38:44.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.328+0000 7fc301000640 1 -- 172.21.15.131:0/816130303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2fc069350 con 0x7fc2fc158770 2024-04-28T12:38:44.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.329+0000 7fc2f2000640 1 -- 172.21.15.131:0/816130303 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2f401c070 con 0x7fc2fc158770 2024-04-28T12:38:44.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.330+0000 7fc2f2000640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc2dc06c920 0x7fc2dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.330+0000 7fc2f2000640 1 -- 172.21.15.131:0/816130303 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc2f408b810 con 0x7fc2fc158770 2024-04-28T12:38:44.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.331+0000 7fc2f9e00640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc2dc06c920 0x7fc2dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.332+0000 7fc2f9e00640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc2dc06c920 0x7fc2dc06ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec033000 tx=0x7fc2ec008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:44.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.334+0000 7fc2f2000640 1 -- 172.21.15.131:0/816130303 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc2f4059840 con 0x7fc2fc158770 2024-04-28T12:38:44.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.541+0000 7fc301000640 1 -- 172.21.15.131:0/816130303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2fc074dd0 con 0x7fc2fc158770 2024-04-28T12:38:44.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.544+0000 7fc2f2000640 1 -- 172.21.15.131:0/816130303 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7fc2f4059140 con 0x7fc2fc158770 2024-04-28T12:38:44.541 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:44.542 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.547+0000 7fc301000640 1 -- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc2dc06c920 msgr2=0x7fc2dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.547+0000 7fc301000640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc2dc06c920 0x7fc2dc06ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec033000 tx=0x7fc2ec008030 comp rx=0 tx=0).stop 2024-04-28T12:38:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.547+0000 7fc301000640 1 -- 172.21.15.131:0/816130303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2fc158770 msgr2=0x7fc2fc1570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.547+0000 7fc301000640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2fc158770 0x7fc2fc1570f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009820 tx=0x7fc2f4009ce0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.547+0000 7fc301000640 1 -- 172.21.15.131:0/816130303 shutdown_connections 2024-04-28T12:38:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.547+0000 7fc301000640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc2dc06c920 0x7fc2dc06ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.547+0000 7fc301000640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2fc158770 0x7fc2fc1570f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.547+0000 7fc301000640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2fc15b6c0 0x7fc2fc148a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.547+0000 7fc301000640 1 --2- 172.21.15.131:0/816130303 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2e00039b0 0x7fc2fc145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.547+0000 7fc301000640 1 -- 172.21.15.131:0/816130303 >> 172.21.15.131:0/816130303 conn(0x7fc2fc0b2900 msgr2=0x7fc2fc15d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.547+0000 7fc301000640 1 -- 172.21.15.131:0/816130303 shutdown_connections 2024-04-28T12:38:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.547+0000 7fc301000640 1 -- 172.21.15.131:0/816130303 wait complete. 2024-04-28T12:38:44.564 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:44.564 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:44.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.757+0000 7fd80ca00640 1 Processor -- start 2024-04-28T12:38:44.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.757+0000 7fd80ca00640 1 -- start start 2024-04-28T12:38:44.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.758+0000 7fd80ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8080cfc30 0x7fd8080d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.759+0000 7fd80ca00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fd8080d1d70 0x7fd8080d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.759+0000 7fd80ca00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8080d12e0 con 0x7fd80815cca0 2024-04-28T12:38:44.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.759+0000 7fd80ca00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fd8080d21c0 con 0x7fd8080d1d70 2024-04-28T12:38:44.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.759+0000 7fd80ca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd8080d0240 con 0x7fd8080cfc30 2024-04-28T12:38:44.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.759+0000 7fd805e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd80815cca0 0x7fd808160240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49352/0 (socket says 172.21.15.131:49352) 2024-04-28T12:38:44.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.759+0000 7fd805e00640 1 -- 172.21.15.131:0/3262338009 learned_addr learned my addr 172.21.15.131:0/3262338009 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:44.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.759+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 990638041 0 0) 0x7fd7f8001110 con 0x7fd80815cca0 2024-04-28T12:38:44.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.759+0000 7fd806800640 1 --2- 172.21.15.131:0/3262338009 >> v2:172.21.15.157:3300/0 conn(0x7fd8080d1d70 0x7fd8080d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.759+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7ec003520 con 0x7fd80815cca0 2024-04-28T12:38:44.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.759+0000 7fd805400640 1 --2- 172.21.15.131:0/3262338009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8080cfc30 0x7fd8080d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2293774961 0 0) 0x7fd7ec003520 con 0x7fd80815cca0 2024-04-28T12:38:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8080cfc30 msgr2=0x7fd8080d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 --2- 172.21.15.131:0/3262338009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8080cfc30 0x7fd8080d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 >> v2:172.21.15.157:3300/0 conn(0x7fd8080d1d70 msgr2=0x7fd8080d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 --2- 172.21.15.131:0/3262338009 >> v2:172.21.15.157:3300/0 conn(0x7fd8080d1d70 0x7fd8080d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8080d0520 con 0x7fd80815cca0 2024-04-28T12:38:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fd7f8001840 con 0x7fd80815cca0 2024-04-28T12:38:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 >> v1:172.21.15.131:6789/0 conn(0x7fd80815cca0 legacy=0x7fd808160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 --2- 172.21.15.131:0/3262338009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7ec0039b0 0x7fd7ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 --2- 172.21.15.131:0/3262338009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8080cfc30 0x7fd7ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 --2- 172.21.15.131:0/3262338009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7ec004d40 0x7fd7ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd8080d21c0 con 0x7fd7ec0039b0 2024-04-28T12:38:44.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8080d0240 con 0x7fd7ec004d40 2024-04-28T12:38:44.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd7ec00bb20 con 0x7fd8080cfc30 2024-04-28T12:38:44.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd805e00640 1 --2- 172.21.15.131:0/3262338009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7ec0039b0 0x7fd7ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd805e00640 1 -- 172.21.15.131:0/3262338009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8080cfc30 msgr2=0x7fd7ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:44.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd805e00640 1 --2- 172.21.15.131:0/3262338009 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8080cfc30 0x7fd7ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd805e00640 1 -- 172.21.15.131:0/3262338009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7ec004d40 msgr2=0x7fd7ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:44.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd805e00640 1 --2- 172.21.15.131:0/3262338009 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7ec004d40 0x7fd7ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.760+0000 7fd805e00640 1 -- 172.21.15.131:0/3262338009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ec003520 con 0x7fd7ec0039b0 2024-04-28T12:38:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.761+0000 7fd805e00640 1 --2- 172.21.15.131:0/3262338009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7ec0039b0 0x7fd7ec003dd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c320 tx=0x7fd7ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d31de6bd53deb98 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.761+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8005ad0 con 0x7fd7ec0039b0 2024-04-28T12:38:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.762+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f8034070 con 0x7fd7ec0039b0 2024-04-28T12:38:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.762+0000 7fd804a00640 1 -- 172.21.15.131:0/3262338009 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8007070 con 0x7fd7ec0039b0 2024-04-28T12:38:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.762+0000 7fd80ca00640 1 -- 172.21.15.131:0/3262338009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7ec0039b0 msgr2=0x7fd7ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.762+0000 7fd80ca00640 1 --2- 172.21.15.131:0/3262338009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7ec0039b0 0x7fd7ec003dd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c320 tx=0x7fd7ec005830 comp rx=0 tx=0).stop 2024-04-28T12:38:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.762+0000 7fd806800640 1 -- 172.21.15.131:0/3262338009 reap_dead start 2024-04-28T12:38:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.762+0000 7fd80ca00640 1 -- 172.21.15.131:0/3262338009 shutdown_connections 2024-04-28T12:38:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.762+0000 7fd80ca00640 1 -- 172.21.15.131:0/3262338009 >> 172.21.15.131:0/3262338009 conn(0x7fd8080d4e40 msgr2=0x7fd8080d5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.762+0000 7fd80ca00640 1 -- 172.21.15.131:0/3262338009 shutdown_connections 2024-04-28T12:38:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.762+0000 7fd80ca00640 1 -- 172.21.15.131:0/3262338009 wait complete. 2024-04-28T12:38:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.764+0000 7fd80ca00640 1 Processor -- start 2024-04-28T12:38:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.764+0000 7fd80ca00640 1 -- start start 2024-04-28T12:38:44.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd80ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7ec0039b0 0x7fd8080b9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd80ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8080ccfd0 0x7fd808158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd80ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8080c8950 0x7fd8080c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd80ca00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd8080d0dc0 con 0x7fd7ec0039b0 2024-04-28T12:38:44.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd80ca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8080d0ae0 con 0x7fd8080c8950 2024-04-28T12:38:44.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd80ca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd80815d080 con 0x7fd8080ccfd0 2024-04-28T12:38:44.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd805e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7ec0039b0 0x7fd8080b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd805e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7ec0039b0 0x7fd8080b9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42254/0 (socket says 172.21.15.131:42254) 2024-04-28T12:38:44.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd805e00640 1 -- 172.21.15.131:0/2957357547 learned_addr learned my addr 172.21.15.131:0/2957357547 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd805400640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8080ccfd0 0x7fd808158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd805e00640 1 -- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8080ccfd0 msgr2=0x7fd808158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd805e00640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8080ccfd0 0x7fd808158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd805e00640 1 -- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8080c8950 msgr2=0x7fd8080c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd805e00640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8080c8950 0x7fd8080c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.765+0000 7fd805e00640 1 -- 172.21.15.131:0/2957357547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8080b98d0 con 0x7fd7ec0039b0 2024-04-28T12:38:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.766+0000 7fd805e00640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7ec0039b0 0x7fd8080b9080 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c440 tx=0x7fd8080d06f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.766+0000 7fd7fe000640 1 -- 172.21.15.131:0/2957357547 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8007270 con 0x7fd7ec0039b0 2024-04-28T12:38:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.766+0000 7fd7fe000640 1 -- 172.21.15.131:0/2957357547 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f8007e50 con 0x7fd7ec0039b0 2024-04-28T12:38:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.766+0000 7fd7fe000640 1 -- 172.21.15.131:0/2957357547 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8007630 con 0x7fd7ec0039b0 2024-04-28T12:38:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.766+0000 7fd80ca00640 1 -- 172.21.15.131:0/2957357547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8080b9bb0 con 0x7fd7ec0039b0 2024-04-28T12:38:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.766+0000 7fd80ca00640 1 -- 172.21.15.131:0/2957357547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8080ba170 con 0x7fd7ec0039b0 2024-04-28T12:38:44.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.767+0000 7fd80ca00640 1 -- 172.21.15.131:0/2957357547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd808069350 con 0x7fd7ec0039b0 2024-04-28T12:38:44.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.768+0000 7fd7fe000640 1 -- 172.21.15.131:0/2957357547 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd7f8044440 con 0x7fd7ec0039b0 2024-04-28T12:38:44.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.769+0000 7fd7fe000640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd7e806c920 0x7fd7e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:44.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.769+0000 7fd7fe000640 1 -- 172.21.15.131:0/2957357547 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd7f80ac5b0 con 0x7fd7ec0039b0 2024-04-28T12:38:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.770+0000 7fd805400640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd7e806c920 0x7fd7e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.771+0000 7fd805400640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd7e806c920 0x7fd7e806ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd808159920 tx=0x7fd7f4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:44.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.773+0000 7fd7fe000640 1 -- 172.21.15.131:0/2957357547 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8080d12e0 con 0x7fd7ec0039b0 2024-04-28T12:38:44.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.970+0000 7fd80ca00640 1 -- 172.21.15.131:0/2957357547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8080c84d0 con 0x7fd7ec0039b0 2024-04-28T12:38:44.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.971+0000 7fd7fe000640 1 -- 172.21.15.131:0/2957357547 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fd7f807a790 con 0x7fd7ec0039b0 2024-04-28T12:38:44.968 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.974+0000 7fd80ca00640 1 -- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd7e806c920 msgr2=0x7fd7e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.974+0000 7fd80ca00640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd7e806c920 0x7fd7e806ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd808159920 tx=0x7fd7f4008040 comp rx=0 tx=0).stop 2024-04-28T12:38:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.974+0000 7fd80ca00640 1 -- 172.21.15.131:0/2957357547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7ec0039b0 msgr2=0x7fd8080b9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.974+0000 7fd80ca00640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7ec0039b0 0x7fd8080b9080 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c440 tx=0x7fd8080d06f0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.974+0000 7fd80ca00640 1 -- 172.21.15.131:0/2957357547 shutdown_connections 2024-04-28T12:38:44.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.975+0000 7fd80ca00640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd7e806c920 0x7fd7e806ede0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.975+0000 7fd80ca00640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8080c8950 0x7fd8080c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.975+0000 7fd80ca00640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd8080ccfd0 0x7fd808158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.975+0000 7fd80ca00640 1 --2- 172.21.15.131:0/2957357547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7ec0039b0 0x7fd8080b9080 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:44.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.975+0000 7fd80ca00640 1 -- 172.21.15.131:0/2957357547 >> 172.21.15.131:0/2957357547 conn(0x7fd8080d4e40 msgr2=0x7fd80815d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:44.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.975+0000 7fd80ca00640 1 -- 172.21.15.131:0/2957357547 shutdown_connections 2024-04-28T12:38:44.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:44.975+0000 7fd80ca00640 1 -- 172.21.15.131:0/2957357547 wait complete. 2024-04-28T12:38:44.990 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.196+0000 7fe243e00640 1 Processor -- start 2024-04-28T12:38:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.196+0000 7fe243e00640 1 -- start start 2024-04-28T12:38:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.198+0000 7fe243e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe23c0b5250 0x7fe23c0cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.198+0000 7fe243e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c15d5f0 0x7fe23c15da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.198+0000 7fe243e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe23c06eaf0 con 0x7fe23c0cf440 2024-04-28T12:38:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.198+0000 7fe243e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe23c06f420 con 0x7fe23c0b5250 2024-04-28T12:38:45.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.198+0000 7fe243e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe23c06edd0 con 0x7fe23c15d5f0 2024-04-28T12:38:45.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.198+0000 7fe240a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe23c0b5250 0x7fe23c0cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:45.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.198+0000 7fe241e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c15d5f0 0x7fe23c15da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:45.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.198+0000 7fe241e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c15d5f0 0x7fe23c15da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:45012/0 (socket says 172.21.15.131:45012) 2024-04-28T12:38:45.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.198+0000 7fe241e00640 1 -- 172.21.15.131:0/168180275 learned_addr learned my addr 172.21.15.131:0/168180275 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:45.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.198+0000 7fe240a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe23c0b5250 0x7fe23c0cff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:53700/0 (socket says 172.21.15.131:53700) 2024-04-28T12:38:45.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.199+0000 7fe240a00640 1 -- 172.21.15.131:0/168180275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c15d5f0 msgr2=0x7fe23c15da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:45.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.199+0000 7fe240a00640 1 --2- 172.21.15.131:0/168180275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c15d5f0 0x7fe23c15da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.199+0000 7fe240a00640 1 -- 172.21.15.131:0/168180275 >> v1:172.21.15.157:6789/0 conn(0x7fe23c0cf440 legacy=0x7fe23c0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:45.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.199+0000 7fe240a00640 1 -- 172.21.15.131:0/168180275 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c06f0b0 con 0x7fe23c0b5250 2024-04-28T12:38:45.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.199+0000 7fe241e00640 1 --2- 172.21.15.131:0/168180275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c15d5f0 0x7fe23c15da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:38:45.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.200+0000 7fe240a00640 1 --2- 172.21.15.131:0/168180275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c0b5250 0x7fe23c0cff30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe2280097a0 tx=0x7fe22802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88db1a2cadbcae49 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:45.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.201+0000 7fe23be00640 1 -- 172.21.15.131:0/168180275 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228037070 con 0x7fe23c0b5250 2024-04-28T12:38:45.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.201+0000 7fe23be00640 1 -- 172.21.15.131:0/168180275 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe22802ae50 con 0x7fe23c0b5250 2024-04-28T12:38:45.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.201+0000 7fe23be00640 1 -- 172.21.15.131:0/168180275 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228032660 con 0x7fe23c0b5250 2024-04-28T12:38:45.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.202+0000 7fe243e00640 1 -- 172.21.15.131:0/168180275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c0b5250 msgr2=0x7fe23c0cff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:45.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.202+0000 7fe243e00640 1 --2- 172.21.15.131:0/168180275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c0b5250 0x7fe23c0cff30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe2280097a0 tx=0x7fe22802a100 comp rx=0 tx=0).stop 2024-04-28T12:38:45.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.202+0000 7fe243e00640 1 -- 172.21.15.131:0/168180275 shutdown_connections 2024-04-28T12:38:45.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.202+0000 7fe243e00640 1 --2- 172.21.15.131:0/168180275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c15d5f0 0x7fe23c15da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.202+0000 7fe243e00640 1 --2- 172.21.15.131:0/168180275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c0b5250 0x7fe23c0cff30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.202+0000 7fe243e00640 1 -- 172.21.15.131:0/168180275 >> 172.21.15.131:0/168180275 conn(0x7fe23c0b2900 msgr2=0x7fe23c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:45.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.202+0000 7fe243e00640 1 -- 172.21.15.131:0/168180275 shutdown_connections 2024-04-28T12:38:45.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.202+0000 7fe243e00640 1 -- 172.21.15.131:0/168180275 wait complete. 2024-04-28T12:38:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.204+0000 7fe243e00640 1 Processor -- start 2024-04-28T12:38:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.204+0000 7fe243e00640 1 -- start start 2024-04-28T12:38:45.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.205+0000 7fe243e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c0b5250 0x7fe23c13e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:45.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.205+0000 7fe243e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe23c0cf440 0x7fe23c1f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.205+0000 7fe243e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15d5f0 0x7fe23c1f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.205+0000 7fe243e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c0d0470 con 0x7fe23c0cf440 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.205+0000 7fe243e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c06f8b0 con 0x7fe23c15d5f0 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.205+0000 7fe243e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe23c15dfc0 con 0x7fe23c0b5250 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.205+0000 7fe240a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe23c0cf440 0x7fe23c1f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.205+0000 7fe240a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe23c0cf440 0x7fe23c1f51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42264/0 (socket says 172.21.15.131:42264) 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.205+0000 7fe240a00640 1 -- 172.21.15.131:0/2410805889 learned_addr learned my addr 172.21.15.131:0/2410805889 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.206+0000 7fe241e00640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15d5f0 0x7fe23c1f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.206+0000 7fe241400640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c0b5250 0x7fe23c13e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.206+0000 7fe240a00640 1 -- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c0b5250 msgr2=0x7fe23c13e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.206+0000 7fe240a00640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c0b5250 0x7fe23c13e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.206+0000 7fe240a00640 1 -- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15d5f0 msgr2=0x7fe23c1f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.206+0000 7fe240a00640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15d5f0 0x7fe23c1f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.206+0000 7fe240a00640 1 -- 172.21.15.131:0/2410805889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c141220 con 0x7fe23c0cf440 2024-04-28T12:38:45.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.206+0000 7fe240a00640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe23c0cf440 0x7fe23c1f51d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe2280098c0 tx=0x7fe2280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:45.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.207+0000 7fe239600640 1 -- 172.21.15.131:0/2410805889 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228041020 con 0x7fe23c0cf440 2024-04-28T12:38:45.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.207+0000 7fe239600640 1 -- 172.21.15.131:0/2410805889 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe22803c670 con 0x7fe23c0cf440 2024-04-28T12:38:45.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.207+0000 7fe239600640 1 -- 172.21.15.131:0/2410805889 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228037040 con 0x7fe23c0cf440 2024-04-28T12:38:45.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.207+0000 7fe243e00640 1 -- 172.21.15.131:0/2410805889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23c0725d0 con 0x7fe23c0cf440 2024-04-28T12:38:45.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.207+0000 7fe243e00640 1 -- 172.21.15.131:0/2410805889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23c155cd0 con 0x7fe23c0cf440 2024-04-28T12:38:45.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.208+0000 7fe243e00640 1 -- 172.21.15.131:0/2410805889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe23c069350 con 0x7fe23c0cf440 2024-04-28T12:38:45.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.209+0000 7fe239600640 1 -- 172.21.15.131:0/2410805889 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe228044020 con 0x7fe23c0cf440 2024-04-28T12:38:45.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.210+0000 7fe239600640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe21c06c8f0 0x7fe21c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:45.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.211+0000 7fe239600640 1 -- 172.21.15.131:0/2410805889 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe2280ac630 con 0x7fe23c0cf440 2024-04-28T12:38:45.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.211+0000 7fe241400640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe21c06c8f0 0x7fe21c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:45.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.212+0000 7fe241400640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe21c06c8f0 0x7fe21c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe230001640 tx=0x7fe230009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:45.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.215+0000 7fe239600640 1 -- 172.21.15.131:0/2410805889 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe22804b030 con 0x7fe23c0cf440 2024-04-28T12:38:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.412+0000 7fe243e00640 1 -- 172.21.15.131:0/2410805889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe23c0b3160 con 0x7fe23c0cf440 2024-04-28T12:38:45.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.416+0000 7fe239600640 1 -- 172.21.15.131:0/2410805889 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fe22807a650 con 0x7fe23c0cf440 2024-04-28T12:38:45.413 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.419+0000 7fe243e00640 1 -- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe21c06c8f0 msgr2=0x7fe21c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.419+0000 7fe243e00640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe21c06c8f0 0x7fe21c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe230001640 tx=0x7fe230009040 comp rx=0 tx=0).stop 2024-04-28T12:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.419+0000 7fe243e00640 1 -- 172.21.15.131:0/2410805889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe23c0cf440 msgr2=0x7fe23c1f51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.419+0000 7fe243e00640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe23c0cf440 0x7fe23c1f51d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe2280098c0 tx=0x7fe2280046c0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.419+0000 7fe243e00640 1 -- 172.21.15.131:0/2410805889 shutdown_connections 2024-04-28T12:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.419+0000 7fe243e00640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe21c06c8f0 0x7fe21c06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.419+0000 7fe243e00640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c15d5f0 0x7fe23c1f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.419+0000 7fe243e00640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe23c0cf440 0x7fe23c1f51d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.419+0000 7fe243e00640 1 --2- 172.21.15.131:0/2410805889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c0b5250 0x7fe23c13e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.419+0000 7fe243e00640 1 -- 172.21.15.131:0/2410805889 >> 172.21.15.131:0/2410805889 conn(0x7fe23c0b2900 msgr2=0x7fe23c143870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.419+0000 7fe243e00640 1 -- 172.21.15.131:0/2410805889 shutdown_connections 2024-04-28T12:38:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.420+0000 7fe243e00640 1 -- 172.21.15.131:0/2410805889 wait complete. 2024-04-28T12:38:45.436 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:45.632 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:38:45.637 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:38:45.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.645+0000 7fc03f000640 1 Processor -- start 2024-04-28T12:38:45.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.645+0000 7fc03f000640 1 -- start start 2024-04-28T12:38:45.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.647+0000 7fc03f000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc0380b59e0 0x7fc0380d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:45.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.647+0000 7fc03f000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc0380b4de0 0x7fc0380b5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:45.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.648+0000 7fc03f000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0380d2c40 con 0x7fc0380b4070 2024-04-28T12:38:45.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.648+0000 7fc03f000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc0380b5750 con 0x7fc0380b59e0 2024-04-28T12:38:45.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.648+0000 7fc03f000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc0380d2f20 con 0x7fc0380b4de0 2024-04-28T12:38:45.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.648+0000 7fc037e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc0380b4070 0x7fc0380cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49364/0 (socket says 172.21.15.131:49364) 2024-04-28T12:38:45.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.648+0000 7fc037e00640 1 -- 172.21.15.131:0/2415972581 learned_addr learned my addr 172.21.15.131:0/2415972581 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:45.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.648+0000 7fc037400640 1 --2- 172.21.15.131:0/2415972581 >> v2:172.21.15.131:3300/0 conn(0x7fc0380b59e0 0x7fc0380d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:45.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.648+0000 7fc036a00640 1 -- 172.21.15.131:0/2415972581 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1483924746 0 0) 0x7fc0380d2c40 con 0x7fc0380b4070 2024-04-28T12:38:45.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.648+0000 7fc037400640 1 -- 172.21.15.131:0/2415972581 >> v2:172.21.15.157:3300/0 conn(0x7fc0380b4de0 msgr2=0x7fc0380b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:45.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.648+0000 7fc037400640 1 --2- 172.21.15.131:0/2415972581 >> v2:172.21.15.157:3300/0 conn(0x7fc0380b4de0 0x7fc0380b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.648+0000 7fc037400640 1 -- 172.21.15.131:0/2415972581 >> v1:172.21.15.131:6789/0 conn(0x7fc0380b4070 legacy=0x7fc0380cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:45.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.648+0000 7fc037400640 1 -- 172.21.15.131:0/2415972581 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0380b48c0 con 0x7fc0380b59e0 2024-04-28T12:38:45.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.649+0000 7fc037400640 1 --2- 172.21.15.131:0/2415972581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0380b59e0 0x7fc0380d1ea0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc03808f2f0 tx=0x7fc02002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bfd5787bb2b72c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:45.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.650+0000 7fc036a00640 1 -- 172.21.15.131:0/2415972581 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020037070 con 0x7fc0380b59e0 2024-04-28T12:38:45.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.650+0000 7fc036a00640 1 -- 172.21.15.131:0/2415972581 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc020032440 con 0x7fc0380b59e0 2024-04-28T12:38:45.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.650+0000 7fc036a00640 1 -- 172.21.15.131:0/2415972581 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020032810 con 0x7fc0380b59e0 2024-04-28T12:38:45.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.650+0000 7fc03f000640 1 -- 172.21.15.131:0/2415972581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0380b59e0 msgr2=0x7fc0380d1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:45.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.650+0000 7fc03f000640 1 --2- 172.21.15.131:0/2415972581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0380b59e0 0x7fc0380d1ea0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc03808f2f0 tx=0x7fc02002a1a0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.650+0000 7fc03f000640 1 -- 172.21.15.131:0/2415972581 shutdown_connections 2024-04-28T12:38:45.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.650+0000 7fc03f000640 1 --2- 172.21.15.131:0/2415972581 >> v2:172.21.15.157:3300/0 conn(0x7fc0380b4de0 0x7fc0380b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.650+0000 7fc03f000640 1 --2- 172.21.15.131:0/2415972581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0380b59e0 0x7fc0380d1ea0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.650+0000 7fc03f000640 1 -- 172.21.15.131:0/2415972581 >> 172.21.15.131:0/2415972581 conn(0x7fc0380d4dc0 msgr2=0x7fc0380d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:45.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.650+0000 7fc03f000640 1 -- 172.21.15.131:0/2415972581 shutdown_connections 2024-04-28T12:38:45.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.651+0000 7fc03f000640 1 -- 172.21.15.131:0/2415972581 wait complete. 2024-04-28T12:38:45.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.653+0000 7fc03f000640 1 Processor -- start 2024-04-28T12:38:45.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.653+0000 7fc03f000640 1 -- start start 2024-04-28T12:38:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc03f000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0380b4070 0x7fc0381f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc03f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0380b4de0 0x7fc0380c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc03f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0380b59e0 0x7fc0380d3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc03f000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc0380d58e0 con 0x7fc0380b4070 2024-04-28T12:38:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc03f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc0380d5c50 con 0x7fc0380b59e0 2024-04-28T12:38:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc03f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc0380d01b0 con 0x7fc0380b4de0 2024-04-28T12:38:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc037e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0380b4070 0x7fc0381f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc037e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0380b4070 0x7fc0381f4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42284/0 (socket says 172.21.15.131:42284) 2024-04-28T12:38:45.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc037e00640 1 -- 172.21.15.131:0/2462715305 learned_addr learned my addr 172.21.15.131:0/2462715305 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc037e00640 1 -- 172.21.15.131:0/2462715305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0380b4de0 msgr2=0x7fc0380c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc037e00640 1 --2- 172.21.15.131:0/2462715305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0380b4de0 0x7fc0380c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc037e00640 1 -- 172.21.15.131:0/2462715305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0380b59e0 msgr2=0x7fc0380d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc037e00640 1 --2- 172.21.15.131:0/2462715305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0380b59e0 0x7fc0380d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.654+0000 7fc037e00640 1 -- 172.21.15.131:0/2462715305 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0380c8a70 con 0x7fc0380b4070 2024-04-28T12:38:45.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.655+0000 7fc037e00640 1 --2- 172.21.15.131:0/2462715305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0380b4070 0x7fc0381f4690 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc02800c7c0 tx=0x7fc02800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.655+0000 7fc02fe00640 1 -- 172.21.15.131:0/2462715305 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028018070 con 0x7fc0380b4070 2024-04-28T12:38:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.655+0000 7fc02fe00640 1 -- 172.21.15.131:0/2462715305 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02800a640 con 0x7fc0380b4070 2024-04-28T12:38:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.655+0000 7fc03f000640 1 -- 172.21.15.131:0/2462715305 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0380c8d50 con 0x7fc0380b4070 2024-04-28T12:38:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.655+0000 7fc02fe00640 1 -- 172.21.15.131:0/2462715305 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02800aa30 con 0x7fc0380b4070 2024-04-28T12:38:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.655+0000 7fc03f000640 1 -- 172.21.15.131:0/2462715305 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0380c97b0 con 0x7fc0380b4070 2024-04-28T12:38:45.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.656+0000 7fc03f000640 1 -- 172.21.15.131:0/2462715305 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0380c8f20 con 0x7fc0380b4070 2024-04-28T12:38:45.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.657+0000 7fc02fe00640 1 -- 172.21.15.131:0/2462715305 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc02801b070 con 0x7fc0380b4070 2024-04-28T12:38:45.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.658+0000 7fc02fe00640 1 --2- 172.21.15.131:0/2462715305 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc01406c8f0 0x7fc01406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:45.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.658+0000 7fc02fe00640 1 -- 172.21.15.131:0/2462715305 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc02808b5d0 con 0x7fc0380b4070 2024-04-28T12:38:45.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.659+0000 7fc037400640 1 --2- 172.21.15.131:0/2462715305 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc01406c8f0 0x7fc01406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:45.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.660+0000 7fc037400640 1 --2- 172.21.15.131:0/2462715305 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc01406c8f0 0x7fc01406edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc020034000 tx=0x7fc02002c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:45.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.662+0000 7fc02fe00640 1 -- 172.21.15.131:0/2462715305 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0380c8f20 con 0x7fc0380b4070 2024-04-28T12:38:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.862+0000 7fc03f000640 1 -- 172.21.15.131:0/2462715305 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc0380b79b0 con 0x7fc0380b4070 2024-04-28T12:38:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.863+0000 7fc02fe00640 1 -- 172.21.15.131:0/2462715305 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fc028059600 con 0x7fc0380b4070 2024-04-28T12:38:45.860 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.866+0000 7fc03f000640 1 -- 172.21.15.131:0/2462715305 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc01406c8f0 msgr2=0x7fc01406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.866+0000 7fc03f000640 1 --2- 172.21.15.131:0/2462715305 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc01406c8f0 0x7fc01406edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc020034000 tx=0x7fc02002c000 comp rx=0 tx=0).stop 2024-04-28T12:38:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.866+0000 7fc03f000640 1 -- 172.21.15.131:0/2462715305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0380b4070 msgr2=0x7fc0381f4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.866+0000 7fc03f000640 1 --2- 172.21.15.131:0/2462715305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0380b4070 0x7fc0381f4690 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc02800c7c0 tx=0x7fc02800cc80 comp rx=0 tx=0).stop 2024-04-28T12:38:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.866+0000 7fc03f000640 1 -- 172.21.15.131:0/2462715305 shutdown_connections 2024-04-28T12:38:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.866+0000 7fc03f000640 1 --2- 172.21.15.131:0/2462715305 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc01406c8f0 0x7fc01406edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.866+0000 7fc03f000640 1 --2- 172.21.15.131:0/2462715305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0380b59e0 0x7fc0380d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.866+0000 7fc03f000640 1 --2- 172.21.15.131:0/2462715305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc0380b4de0 0x7fc0380c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.866+0000 7fc03f000640 1 --2- 172.21.15.131:0/2462715305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0380b4070 0x7fc0381f4690 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.866+0000 7fc03f000640 1 -- 172.21.15.131:0/2462715305 >> 172.21.15.131:0/2462715305 conn(0x7fc0380d4dc0 msgr2=0x7fc0380d51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.866+0000 7fc03f000640 1 -- 172.21.15.131:0/2462715305 shutdown_connections 2024-04-28T12:38:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:45.866+0000 7fc03f000640 1 -- 172.21.15.131:0/2462715305 wait complete. 2024-04-28T12:38:45.882 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:46.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.079+0000 7fe9a7600640 1 Processor -- start 2024-04-28T12:38:46.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.079+0000 7fe9a7600640 1 -- start start 2024-04-28T12:38:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.081+0000 7fe9a7600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9a014d200 0x7fe9a0163b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.081+0000 7fe9a7600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a00723f0 0x7fe9a006ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.081+0000 7fe9a7600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe9a006f1b0 0x7fe9a00b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.081+0000 7fe9a7600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a006df40 con 0x7fe9a006f1b0 2024-04-28T12:38:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.081+0000 7fe9a7600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a0073660 con 0x7fe9a014d200 2024-04-28T12:38:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.081+0000 7fe9a7600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a006e190 con 0x7fe9a00723f0 2024-04-28T12:38:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.081+0000 7fe9a5600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe9a006f1b0 0x7fe9a00b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.081+0000 7fe9a5600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe9a006f1b0 0x7fe9a00b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42300/0 (socket says 172.21.15.131:42300) 2024-04-28T12:38:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.081+0000 7fe9a5600640 1 -- 172.21.15.131:0/2296574869 learned_addr learned my addr 172.21.15.131:0/2296574869 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.081+0000 7fe9a4c00640 1 --2- 172.21.15.131:0/2296574869 >> v2:172.21.15.157:3300/0 conn(0x7fe9a014d200 0x7fe9a0163b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.081+0000 7fe99fe00640 1 --2- 172.21.15.131:0/2296574869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a00723f0 0x7fe9a006ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.082+0000 7fe9a5600640 1 -- 172.21.15.131:0/2296574869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a00723f0 msgr2=0x7fe9a006ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:46.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.082+0000 7fe9a5600640 1 --2- 172.21.15.131:0/2296574869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a00723f0 0x7fe9a006ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.082+0000 7fe9a5600640 1 -- 172.21.15.131:0/2296574869 >> v2:172.21.15.157:3300/0 conn(0x7fe9a014d200 msgr2=0x7fe9a0163b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:46.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.082+0000 7fe9a5600640 1 --2- 172.21.15.131:0/2296574869 >> v2:172.21.15.157:3300/0 conn(0x7fe9a014d200 0x7fe9a0163b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.082+0000 7fe9a5600640 1 -- 172.21.15.131:0/2296574869 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a006e470 con 0x7fe9a006f1b0 2024-04-28T12:38:46.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.083+0000 7fe9a5600640 1 --2- 172.21.15.131:0/2296574869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9a006f1b0 0x7fe9a00b3c00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0071b70 tx=0x7fe99802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39edbbe166073f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:46.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.083+0000 7fe99f400640 1 -- 172.21.15.131:0/2296574869 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998036070 con 0x7fe9a006f1b0 2024-04-28T12:38:46.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.083+0000 7fe99f400640 1 -- 172.21.15.131:0/2296574869 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe998031600 con 0x7fe9a006f1b0 2024-04-28T12:38:46.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.083+0000 7fe99f400640 1 -- 172.21.15.131:0/2296574869 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9980319d0 con 0x7fe9a006f1b0 2024-04-28T12:38:46.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.084+0000 7fe9a7600640 1 -- 172.21.15.131:0/2296574869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9a006f1b0 msgr2=0x7fe9a00b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:46.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.084+0000 7fe9a7600640 1 --2- 172.21.15.131:0/2296574869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9a006f1b0 0x7fe9a00b3c00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0071b70 tx=0x7fe99802a6a0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.084+0000 7fe9a7600640 1 -- 172.21.15.131:0/2296574869 shutdown_connections 2024-04-28T12:38:46.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.084+0000 7fe9a7600640 1 --2- 172.21.15.131:0/2296574869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9a006f1b0 0x7fe9a00b3c00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.084+0000 7fe9a7600640 1 --2- 172.21.15.131:0/2296574869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a00723f0 0x7fe9a006ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.084+0000 7fe9a7600640 1 --2- 172.21.15.131:0/2296574869 >> v2:172.21.15.157:3300/0 conn(0x7fe9a014d200 0x7fe9a0163b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.084+0000 7fe9a7600640 1 -- 172.21.15.131:0/2296574869 >> 172.21.15.131:0/2296574869 conn(0x7fe9a00b2900 msgr2=0x7fe9a00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:46.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.084+0000 7fe9a7600640 1 -- 172.21.15.131:0/2296574869 shutdown_connections 2024-04-28T12:38:46.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.084+0000 7fe9a7600640 1 -- 172.21.15.131:0/2296574869 wait complete. 2024-04-28T12:38:46.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.086+0000 7fe9a7600640 1 Processor -- start 2024-04-28T12:38:46.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.086+0000 7fe9a7600640 1 -- start start 2024-04-28T12:38:46.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.087+0000 7fe9a7600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a006f1b0 0x7fe9a015f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.087+0000 7fe9a7600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9a00723f0 0x7fe9a013b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.087+0000 7fe9a7600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9a014d200 0x7fe9a014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.087+0000 7fe9a7600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a0073440 con 0x7fe9a00723f0 2024-04-28T12:38:46.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.087+0000 7fe9a7600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a00732c0 con 0x7fe9a014d200 2024-04-28T12:38:46.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.087+0000 7fe9a7600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a006f640 con 0x7fe9a006f1b0 2024-04-28T12:38:46.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.088+0000 7fe99fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9a00723f0 0x7fe9a013b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.088+0000 7fe99fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9a00723f0 0x7fe9a013b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42308/0 (socket says 172.21.15.131:42308) 2024-04-28T12:38:46.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.088+0000 7fe99fe00640 1 -- 172.21.15.131:0/1125501953 learned_addr learned my addr 172.21.15.131:0/1125501953 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:46.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.088+0000 7fe9a4c00640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a006f1b0 0x7fe9a015f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.088+0000 7fe9a5600640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9a014d200 0x7fe9a014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.088+0000 7fe99fe00640 1 -- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a006f1b0 msgr2=0x7fe9a015f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:46.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.088+0000 7fe99fe00640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a006f1b0 0x7fe9a015f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.088+0000 7fe99fe00640 1 -- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9a014d200 msgr2=0x7fe9a014b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:46.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.088+0000 7fe99fe00640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9a014d200 0x7fe9a014b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.088+0000 7fe99fe00640 1 -- 172.21.15.131:0/1125501953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a01611c0 con 0x7fe9a00723f0 2024-04-28T12:38:46.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.088+0000 7fe9a5600640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9a014d200 0x7fe9a014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:46.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.089+0000 7fe99fe00640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9a00723f0 0x7fe9a013b8a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00d800 tx=0x7fe98c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:46.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.089+0000 7fe99cc00640 1 -- 172.21.15.131:0/1125501953 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c018020 con 0x7fe9a00723f0 2024-04-28T12:38:46.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.089+0000 7fe99cc00640 1 -- 172.21.15.131:0/1125501953 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe98c0096a0 con 0x7fe9a00723f0 2024-04-28T12:38:46.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.089+0000 7fe9a7600640 1 -- 172.21.15.131:0/1125501953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a01614a0 con 0x7fe9a00723f0 2024-04-28T12:38:46.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.089+0000 7fe99cc00640 1 -- 172.21.15.131:0/1125501953 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c009a70 con 0x7fe9a00723f0 2024-04-28T12:38:46.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.089+0000 7fe9a7600640 1 -- 172.21.15.131:0/1125501953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a0161d40 con 0x7fe9a00723f0 2024-04-28T12:38:46.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.090+0000 7fe9a7600640 1 -- 172.21.15.131:0/1125501953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a0161670 con 0x7fe9a00723f0 2024-04-28T12:38:46.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.091+0000 7fe99cc00640 1 -- 172.21.15.131:0/1125501953 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe98c01b020 con 0x7fe9a00723f0 2024-04-28T12:38:46.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.092+0000 7fe99cc00640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe98006c920 0x7fe98006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.092+0000 7fe99cc00640 1 -- 172.21.15.131:0/1125501953 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe98c08c230 con 0x7fe9a00723f0 2024-04-28T12:38:46.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.093+0000 7fe9a4c00640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe98006c920 0x7fe98006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.094+0000 7fe9a4c00640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe98006c920 0x7fe98006ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe994009a30 tx=0x7fe994008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:46.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.096+0000 7fe99cc00640 1 -- 172.21.15.131:0/1125501953 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe98c022030 con 0x7fe9a00723f0 2024-04-28T12:38:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.279+0000 7fe9a7600640 1 -- 172.21.15.131:0/1125501953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe9a0069350 con 0x7fe9a00723f0 2024-04-28T12:38:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.281+0000 7fe99cc00640 1 -- 172.21.15.131:0/1125501953 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fe98c05a1c0 con 0x7fe9a00723f0 2024-04-28T12:38:46.277 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:46.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.283+0000 7fe9a7600640 1 -- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe98006c920 msgr2=0x7fe98006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:46.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.283+0000 7fe9a7600640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe98006c920 0x7fe98006ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe994009a30 tx=0x7fe994008040 comp rx=0 tx=0).stop 2024-04-28T12:38:46.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.283+0000 7fe9a7600640 1 -- 172.21.15.131:0/1125501953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9a00723f0 msgr2=0x7fe9a013b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:46.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.283+0000 7fe9a7600640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9a00723f0 0x7fe9a013b8a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00d800 tx=0x7fe98c00dcc0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.283+0000 7fe9a7600640 1 -- 172.21.15.131:0/1125501953 shutdown_connections 2024-04-28T12:38:46.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.283+0000 7fe9a7600640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe98006c920 0x7fe98006ede0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.283+0000 7fe9a7600640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9a014d200 0x7fe9a014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.283+0000 7fe9a7600640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9a00723f0 0x7fe9a013b8a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.283+0000 7fe9a7600640 1 --2- 172.21.15.131:0/1125501953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9a006f1b0 0x7fe9a015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.283+0000 7fe9a7600640 1 -- 172.21.15.131:0/1125501953 >> 172.21.15.131:0/1125501953 conn(0x7fe9a00b2900 msgr2=0x7fe9a00b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:46.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.284+0000 7fe9a7600640 1 -- 172.21.15.131:0/1125501953 shutdown_connections 2024-04-28T12:38:46.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.284+0000 7fe9a7600640 1 -- 172.21.15.131:0/1125501953 wait complete. 2024-04-28T12:38:46.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T12:38:46.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.506+0000 7fea7be00640 1 Processor -- start 2024-04-28T12:38:46.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.506+0000 7fea7be00640 1 -- start start 2024-04-28T12:38:46.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.508+0000 7fea7be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fea7c145330 0x7fea7c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.508+0000 7fea7be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea7c1628e0 0x7fea7c162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.508+0000 7fea7be00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea7c145710 con 0x7fea7c0af720 2024-04-28T12:38:46.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.508+0000 7fea7be00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fea7c163280 con 0x7fea7c145330 2024-04-28T12:38:46.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.508+0000 7fea7be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fea7c073dc0 con 0x7fea7c1628e0 2024-04-28T12:38:46.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.508+0000 7fea7aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fea7c145330 0x7fea7c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.508+0000 7fea7aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fea7c145330 0x7fea7c0b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42318/0 (socket says 172.21.15.131:42318) 2024-04-28T12:38:46.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.508+0000 7fea7aa00640 1 -- 172.21.15.131:0/2917628228 learned_addr learned my addr 172.21.15.131:0/2917628228 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:46.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.508+0000 7fea7b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fea7c0af720 0x7fea7c0afbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49372/0 (socket says 172.21.15.131:49372) 2024-04-28T12:38:46.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.509+0000 7fea7a000640 1 --2- 172.21.15.131:0/2917628228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea7c1628e0 0x7fea7c162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.509+0000 7fea7aa00640 1 -- 172.21.15.131:0/2917628228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea7c1628e0 msgr2=0x7fea7c162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:46.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.509+0000 7fea7aa00640 1 --2- 172.21.15.131:0/2917628228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea7c1628e0 0x7fea7c162d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.509+0000 7fea7aa00640 1 -- 172.21.15.131:0/2917628228 >> v1:172.21.15.131:6789/0 conn(0x7fea7c0af720 legacy=0x7fea7c0afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:46.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.509+0000 7fea79600640 1 -- 172.21.15.131:0/2917628228 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2478940477 0 0) 0x7fea7c145710 con 0x7fea7c0af720 2024-04-28T12:38:46.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.509+0000 7fea7aa00640 1 -- 172.21.15.131:0/2917628228 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c0ae960 con 0x7fea7c145330 2024-04-28T12:38:46.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.510+0000 7fea7aa00640 1 --2- 172.21.15.131:0/2917628228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c145330 0x7fea7c0b3940 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009870 tx=0x7fea6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d03c47d70a3b640 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.510+0000 7fea79600640 1 -- 172.21.15.131:0/2917628228 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c037070 con 0x7fea7c145330 2024-04-28T12:38:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.510+0000 7fea79600640 1 -- 172.21.15.131:0/2917628228 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6c032440 con 0x7fea7c145330 2024-04-28T12:38:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.510+0000 7fea79600640 1 -- 172.21.15.131:0/2917628228 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c032810 con 0x7fea7c145330 2024-04-28T12:38:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.511+0000 7fea7be00640 1 -- 172.21.15.131:0/2917628228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c145330 msgr2=0x7fea7c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.511+0000 7fea7be00640 1 --2- 172.21.15.131:0/2917628228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c145330 0x7fea7c0b3940 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009870 tx=0x7fea6c02a270 comp rx=0 tx=0).stop 2024-04-28T12:38:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.511+0000 7fea7be00640 1 -- 172.21.15.131:0/2917628228 shutdown_connections 2024-04-28T12:38:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.511+0000 7fea7be00640 1 --2- 172.21.15.131:0/2917628228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea7c1628e0 0x7fea7c162d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.511+0000 7fea7be00640 1 --2- 172.21.15.131:0/2917628228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c145330 0x7fea7c0b3940 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.511+0000 7fea7be00640 1 -- 172.21.15.131:0/2917628228 >> 172.21.15.131:0/2917628228 conn(0x7fea7c068090 msgr2=0x7fea7c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.511+0000 7fea7be00640 1 -- 172.21.15.131:0/2917628228 shutdown_connections 2024-04-28T12:38:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.511+0000 7fea7be00640 1 -- 172.21.15.131:0/2917628228 wait complete. 2024-04-28T12:38:46.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.512+0000 7fea7be00640 1 Processor -- start 2024-04-28T12:38:46.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.513+0000 7fea7be00640 1 -- start start 2024-04-28T12:38:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.513+0000 7fea7be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7c0af720 0x7fea7c0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c145330 0x7fea7c1f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea7c1628e0 0x7fea7c13fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7be00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c0af200 con 0x7fea7c145330 2024-04-28T12:38:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c0aef20 con 0x7fea7c0af720 2024-04-28T12:38:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fea7c0b3e80 con 0x7fea7c1628e0 2024-04-28T12:38:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c145330 0x7fea7c1f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c145330 0x7fea7c1f5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42330/0 (socket says 172.21.15.131:42330) 2024-04-28T12:38:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7a000640 1 -- 172.21.15.131:0/372757451 learned_addr learned my addr 172.21.15.131:0/372757451 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7aa00640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7c0af720 0x7fea7c0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7b400640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea7c1628e0 0x7fea7c13fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7a000640 1 -- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea7c1628e0 msgr2=0x7fea7c13fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7a000640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea7c1628e0 0x7fea7c13fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7a000640 1 -- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7c0af720 msgr2=0x7fea7c0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7a000640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7c0af720 0x7fea7c0b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7a000640 1 -- 172.21.15.131:0/372757451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c0b21f0 con 0x7fea7c145330 2024-04-28T12:38:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.514+0000 7fea7b400640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea7c1628e0 0x7fea7c13fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.515+0000 7fea7a000640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c145330 0x7fea7c1f5a90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fea6400d5f0 tx=0x7fea6400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:46.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.515+0000 7fea72a00640 1 -- 172.21.15.131:0/372757451 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea640125f0 con 0x7fea7c145330 2024-04-28T12:38:46.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.515+0000 7fea72a00640 1 -- 172.21.15.131:0/372757451 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea64019070 con 0x7fea7c145330 2024-04-28T12:38:46.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.515+0000 7fea72a00640 1 -- 172.21.15.131:0/372757451 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea640129d0 con 0x7fea7c145330 2024-04-28T12:38:46.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.515+0000 7fea7be00640 1 -- 172.21.15.131:0/372757451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c13e370 con 0x7fea7c145330 2024-04-28T12:38:46.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.515+0000 7fea7be00640 1 -- 172.21.15.131:0/372757451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c13e930 con 0x7fea7c145330 2024-04-28T12:38:46.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.516+0000 7fea7be00640 1 -- 172.21.15.131:0/372757451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea7c069350 con 0x7fea7c145330 2024-04-28T12:38:46.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.517+0000 7fea72a00640 1 -- 172.21.15.131:0/372757451 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea64012b70 con 0x7fea7c145330 2024-04-28T12:38:46.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.518+0000 7fea72a00640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fea5806c8f0 0x7fea5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:46.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.518+0000 7fea72a00640 1 -- 172.21.15.131:0/372757451 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fea6408c440 con 0x7fea7c145330 2024-04-28T12:38:46.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.519+0000 7fea7aa00640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fea5806c8f0 0x7fea5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:46.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.520+0000 7fea7aa00640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fea5806c8f0 0x7fea5806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea6c034000 tx=0x7fea6c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:46.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.522+0000 7fea72a00640 1 -- 172.21.15.131:0/372757451 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fea6405a3d0 con 0x7fea7c145330 2024-04-28T12:38:46.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.738+0000 7fea7be00640 1 -- 172.21.15.131:0/372757451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fea7c142a50 con 0x7fea7c145330 2024-04-28T12:38:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.738+0000 7fea72a00640 1 -- 172.21.15.131:0/372757451 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fea64059cd0 con 0x7fea7c145330 2024-04-28T12:38:46.735 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-04-28T12:38:46.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.741+0000 7fea7be00640 1 -- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fea5806c8f0 msgr2=0x7fea5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:46.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.741+0000 7fea7be00640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fea5806c8f0 0x7fea5806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea6c034000 tx=0x7fea6c02e000 comp rx=0 tx=0).stop 2024-04-28T12:38:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.741+0000 7fea7be00640 1 -- 172.21.15.131:0/372757451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c145330 msgr2=0x7fea7c1f5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.741+0000 7fea7be00640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c145330 0x7fea7c1f5a90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fea6400d5f0 tx=0x7fea6400dab0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.742+0000 7fea7be00640 1 -- 172.21.15.131:0/372757451 shutdown_connections 2024-04-28T12:38:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.742+0000 7fea7be00640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fea5806c8f0 0x7fea5806edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.742+0000 7fea7be00640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea7c1628e0 0x7fea7c13fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.742+0000 7fea7be00640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea7c145330 0x7fea7c1f5a90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.742+0000 7fea7be00640 1 --2- 172.21.15.131:0/372757451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea7c0af720 0x7fea7c0b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.742+0000 7fea7be00640 1 -- 172.21.15.131:0/372757451 >> 172.21.15.131:0/372757451 conn(0x7fea7c068090 msgr2=0x7fea7c15cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.742+0000 7fea7be00640 1 -- 172.21.15.131:0/372757451 shutdown_connections 2024-04-28T12:38:46.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:46.742+0000 7fea7be00640 1 -- 172.21.15.131:0/372757451 wait complete. 2024-04-28T12:38:46.758 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-04-28T12:38:46.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:46.758 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-28T12:38:46.839 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:38:46.839 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:38:46.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:46.840 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-28T12:38:46.890 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:38:46.890 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:38:46.947 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:38:46.948 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:38:46.998 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:47.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.212+0000 7fc81de00640 1 Processor -- start 2024-04-28T12:38:47.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.212+0000 7fc81de00640 1 -- start start 2024-04-28T12:38:47.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.214+0000 7fc81de00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc8181628e0 0x7fc818162d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:47.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.214+0000 7fc81de00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc818145710 con 0x7fc8180af720 2024-04-28T12:38:47.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.214+0000 7fc81de00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc818073dc0 con 0x7fc818145330 2024-04-28T12:38:47.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.214+0000 7fc81de00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc8180b0250 con 0x7fc8181628e0 2024-04-28T12:38:47.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.214+0000 7fc80fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc8181628e0 0x7fc818162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:47.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.214+0000 7fc81d400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc8180af720 0x7fc8180afb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49378/0 (socket says 172.21.15.131:49378) 2024-04-28T12:38:47.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.214+0000 7fc81d400640 1 -- 172.21.15.131:0/3968582868 learned_addr learned my addr 172.21.15.131:0/3968582868 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:47.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.215+0000 7fc80fe00640 1 -- 172.21.15.131:0/3968582868 >> v1:172.21.15.157:6789/0 conn(0x7fc818145330 legacy=0x7fc8180b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:47.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.215+0000 7fc80fe00640 1 -- 172.21.15.131:0/3968582868 >> v1:172.21.15.131:6789/0 conn(0x7fc8180af720 legacy=0x7fc8180afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:47.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.215+0000 7fc80fe00640 1 -- 172.21.15.131:0/3968582868 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8180ae960 con 0x7fc8181628e0 2024-04-28T12:38:47.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.215+0000 7fc817e00640 1 -- 172.21.15.131:0/3968582868 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 961670458 0 0) 0x7fc818145710 con 0x7fc8180af720 2024-04-28T12:38:47.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.216+0000 7fc80fe00640 1 --2- 172.21.15.131:0/3968582868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8181628e0 0x7fc818162d70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc8180b03d0 tx=0x7fc80002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7918211e5a18847 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:47.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.216+0000 7fc817e00640 1 -- 172.21.15.131:0/3968582868 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800037070 con 0x7fc8181628e0 2024-04-28T12:38:47.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.216+0000 7fc817e00640 1 -- 172.21.15.131:0/3968582868 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc800032440 con 0x7fc8181628e0 2024-04-28T12:38:47.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.216+0000 7fc817e00640 1 -- 172.21.15.131:0/3968582868 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800032810 con 0x7fc8181628e0 2024-04-28T12:38:47.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.217+0000 7fc81de00640 1 -- 172.21.15.131:0/3968582868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8181628e0 msgr2=0x7fc818162d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.217+0000 7fc81de00640 1 --2- 172.21.15.131:0/3968582868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8181628e0 0x7fc818162d70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc8180b03d0 tx=0x7fc80002a1c0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.217+0000 7fc81de00640 1 -- 172.21.15.131:0/3968582868 shutdown_connections 2024-04-28T12:38:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.217+0000 7fc81de00640 1 --2- 172.21.15.131:0/3968582868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8181628e0 0x7fc818162d70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.217+0000 7fc81de00640 1 -- 172.21.15.131:0/3968582868 >> 172.21.15.131:0/3968582868 conn(0x7fc818068090 msgr2=0x7fc818068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.217+0000 7fc81de00640 1 -- 172.21.15.131:0/3968582868 shutdown_connections 2024-04-28T12:38:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.217+0000 7fc81de00640 1 -- 172.21.15.131:0/3968582868 wait complete. 2024-04-28T12:38:47.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.219+0000 7fc81de00640 1 Processor -- start 2024-04-28T12:38:47.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.219+0000 7fc81de00640 1 -- start start 2024-04-28T12:38:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.220+0000 7fc81de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8180af720 0x7fc81815c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.220+0000 7fc81de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc818145330 0x7fc8180b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.220+0000 7fc81de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8181628e0 0x7fc818142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.220+0000 7fc81de00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc8180af200 con 0x7fc8180af720 2024-04-28T12:38:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.220+0000 7fc81de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc8180aef20 con 0x7fc8181628e0 2024-04-28T12:38:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.220+0000 7fc81de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc818073000 con 0x7fc818145330 2024-04-28T12:38:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.220+0000 7fc81ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8180af720 0x7fc81815c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.220+0000 7fc81ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8180af720 0x7fc81815c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42362/0 (socket says 172.21.15.131:42362) 2024-04-28T12:38:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.220+0000 7fc81ca00640 1 -- 172.21.15.131:0/3410174923 learned_addr learned my addr 172.21.15.131:0/3410174923 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.220+0000 7fc81d400640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8181628e0 0x7fc818142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.220+0000 7fc81ca00640 1 -- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc818145330 msgr2=0x7fc8180b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:47.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.220+0000 7fc80fe00640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc818145330 0x7fc8180b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:47.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.221+0000 7fc81ca00640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc818145330 0x7fc8180b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.221+0000 7fc81ca00640 1 -- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8181628e0 msgr2=0x7fc818142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:47.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.221+0000 7fc81ca00640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8181628e0 0x7fc818142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.221+0000 7fc81ca00640 1 -- 172.21.15.131:0/3410174923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8180b0eb0 con 0x7fc8180af720 2024-04-28T12:38:47.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.221+0000 7fc81ca00640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8180af720 0x7fc81815c160 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc80800c430 tx=0x7fc80800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:47.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.221+0000 7fc815600640 1 -- 172.21.15.131:0/3410174923 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808011790 con 0x7fc8180af720 2024-04-28T12:38:47.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.221+0000 7fc815600640 1 -- 172.21.15.131:0/3410174923 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc808017070 con 0x7fc8180af720 2024-04-28T12:38:47.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.221+0000 7fc81de00640 1 -- 172.21.15.131:0/3410174923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8180b1190 con 0x7fc8180af720 2024-04-28T12:38:47.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.221+0000 7fc815600640 1 -- 172.21.15.131:0/3410174923 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808011bb0 con 0x7fc8180af720 2024-04-28T12:38:47.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.221+0000 7fc81de00640 1 -- 172.21.15.131:0/3410174923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8180b1750 con 0x7fc8180af720 2024-04-28T12:38:47.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.223+0000 7fc81de00640 1 -- 172.21.15.131:0/3410174923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc818069350 con 0x7fc8180af720 2024-04-28T12:38:47.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.223+0000 7fc815600640 1 -- 172.21.15.131:0/3410174923 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc80801c460 con 0x7fc8180af720 2024-04-28T12:38:47.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.224+0000 7fc815600640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7f406c8f0 0x7fc7f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:47.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.224+0000 7fc815600640 1 -- 172.21.15.131:0/3410174923 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc80808be90 con 0x7fc8180af720 2024-04-28T12:38:47.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.225+0000 7fc80fe00640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7f406c8f0 0x7fc7f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:47.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.226+0000 7fc80fe00640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7f406c8f0 0x7fc7f406edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc800034000 tx=0x7fc80002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:47.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.229+0000 7fc815600640 1 -- 172.21.15.131:0/3410174923 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc808059e20 con 0x7fc8180af720 2024-04-28T12:38:47.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.428+0000 7fc81de00640 1 -- 172.21.15.131:0/3410174923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8181429c0 con 0x7fc8180af720 2024-04-28T12:38:47.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.432+0000 7fc815600640 1 -- 172.21.15.131:0/3410174923 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7fc80808d020 con 0x7fc8180af720 2024-04-28T12:38:47.429 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:47.429 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.435+0000 7fc81de00640 1 -- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7f406c8f0 msgr2=0x7fc7f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.435+0000 7fc81de00640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7f406c8f0 0x7fc7f406edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc800034000 tx=0x7fc80002e000 comp rx=0 tx=0).stop 2024-04-28T12:38:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.435+0000 7fc81de00640 1 -- 172.21.15.131:0/3410174923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8180af720 msgr2=0x7fc81815c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.435+0000 7fc81de00640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8180af720 0x7fc81815c160 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc80800c430 tx=0x7fc80800c8f0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.435+0000 7fc81de00640 1 -- 172.21.15.131:0/3410174923 shutdown_connections 2024-04-28T12:38:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.435+0000 7fc81de00640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7f406c8f0 0x7fc7f406edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.435+0000 7fc81de00640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8181628e0 0x7fc818142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.435+0000 7fc81de00640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc818145330 0x7fc8180b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.435+0000 7fc81de00640 1 --2- 172.21.15.131:0/3410174923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8180af720 0x7fc81815c160 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.435+0000 7fc81de00640 1 -- 172.21.15.131:0/3410174923 >> 172.21.15.131:0/3410174923 conn(0x7fc818068090 msgr2=0x7fc8180afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.435+0000 7fc81de00640 1 -- 172.21.15.131:0/3410174923 shutdown_connections 2024-04-28T12:38:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.435+0000 7fc81de00640 1 -- 172.21.15.131:0/3410174923 wait complete. 2024-04-28T12:38:47.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:47.453 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:47.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.663+0000 7f2ae9000640 1 Processor -- start 2024-04-28T12:38:47.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.663+0000 7f2ae9000640 1 -- start start 2024-04-28T12:38:47.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.665+0000 7f2ae9000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2ae4155cd0 0x7f2ae4072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:47.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.665+0000 7f2ae9000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2ae40b5b10 0x7f2ae4162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:47.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.665+0000 7f2ae9000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae415f920 con 0x7f2ae4072b40 2024-04-28T12:38:47.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.665+0000 7f2ae9000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae415fc00 con 0x7f2ae40b5b10 2024-04-28T12:38:47.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.665+0000 7f2ae9000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae415fee0 con 0x7f2ae4155cd0 2024-04-28T12:38:47.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.665+0000 7f2ae3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2ae40b5b10 0x7f2ae4162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:47.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.665+0000 7f2ae2a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2ae4072b40 0x7f2ae40b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49390/0 (socket says 172.21.15.131:49390) 2024-04-28T12:38:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.665+0000 7f2ae2a00640 1 -- 172.21.15.131:0/3664171921 learned_addr learned my addr 172.21.15.131:0/3664171921 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.665+0000 7f2ae3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2ae40b5b10 0x7f2ae4162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42374/0 (socket says 172.21.15.131:42374) 2024-04-28T12:38:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.665+0000 7f2ae3400640 1 --2- 172.21.15.131:0/3664171921 >> v2:172.21.15.157:3300/0 conn(0x7f2ae4155cd0 0x7f2ae4072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.666+0000 7f2ae3e00640 1 -- 172.21.15.131:0/3664171921 >> v2:172.21.15.157:3300/0 conn(0x7f2ae4155cd0 msgr2=0x7f2ae4072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.666+0000 7f2ae3e00640 1 --2- 172.21.15.131:0/3664171921 >> v2:172.21.15.157:3300/0 conn(0x7f2ae4155cd0 0x7f2ae4072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.666+0000 7f2ae3e00640 1 -- 172.21.15.131:0/3664171921 >> v1:172.21.15.131:6789/0 conn(0x7f2ae4072b40 legacy=0x7f2ae40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.666+0000 7f2ae2000640 1 -- 172.21.15.131:0/3664171921 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1594160006 0 0) 0x7f2ae415f920 con 0x7f2ae4072b40 2024-04-28T12:38:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.666+0000 7f2ae3e00640 1 -- 172.21.15.131:0/3664171921 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae41601c0 con 0x7f2ae40b5b10 2024-04-28T12:38:47.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.666+0000 7f2ae3400640 1 --2- 172.21.15.131:0/3664171921 >> v2:172.21.15.157:3300/0 conn(0x7f2ae4155cd0 0x7f2ae4072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:47.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.667+0000 7f2ae3e00640 1 --2- 172.21.15.131:0/3664171921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae40b5b10 0x7f2ae4162f00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40d1320 tx=0x7f2adc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6712913e4e3f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.667+0000 7f2ae2000640 1 -- 172.21.15.131:0/3664171921 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc007740 con 0x7f2ae40b5b10 2024-04-28T12:38:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.667+0000 7f2ae2000640 1 -- 172.21.15.131:0/3664171921 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc0078e0 con 0x7f2ae40b5b10 2024-04-28T12:38:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.667+0000 7f2ae2000640 1 -- 172.21.15.131:0/3664171921 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc007cb0 con 0x7f2ae40b5b10 2024-04-28T12:38:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.667+0000 7f2ae9000640 1 -- 172.21.15.131:0/3664171921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae40b5b10 msgr2=0x7f2ae4162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.667+0000 7f2ae9000640 1 --2- 172.21.15.131:0/3664171921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae40b5b10 0x7f2ae4162f00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40d1320 tx=0x7f2adc02a710 comp rx=0 tx=0).stop 2024-04-28T12:38:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.668+0000 7f2ae9000640 1 -- 172.21.15.131:0/3664171921 shutdown_connections 2024-04-28T12:38:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.668+0000 7f2ae9000640 1 --2- 172.21.15.131:0/3664171921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae40b5b10 0x7f2ae4162f00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.668+0000 7f2ae9000640 1 --2- 172.21.15.131:0/3664171921 >> v2:172.21.15.157:3300/0 conn(0x7f2ae4155cd0 0x7f2ae4072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.668+0000 7f2ae9000640 1 -- 172.21.15.131:0/3664171921 >> 172.21.15.131:0/3664171921 conn(0x7f2ae4068090 msgr2=0x7f2ae40684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.668+0000 7f2ae9000640 1 -- 172.21.15.131:0/3664171921 shutdown_connections 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.668+0000 7f2ae9000640 1 -- 172.21.15.131:0/3664171921 wait complete. 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.669+0000 7f2ae9000640 1 Processor -- start 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.670+0000 7f2ae9000640 1 -- start start 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.670+0000 7f2ae9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ae4072b40 0x7f2ae4159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae9000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ae40b5b10 0x7f2ae4151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae9000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4155cd0 0x7f2ae4161550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae9000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae4156230 con 0x7f2ae4155cd0 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae9000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41560b0 con 0x7f2ae4072b40 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae9000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae415f3f0 con 0x7f2ae40b5b10 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4155cd0 0x7f2ae4161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4155cd0 0x7f2ae4161550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42378/0 (socket says 172.21.15.131:42378) 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae3e00640 1 -- 172.21.15.131:0/1456816126 learned_addr learned my addr 172.21.15.131:0/1456816126 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae3400640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ae4072b40 0x7f2ae4159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae3e00640 1 -- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ae40b5b10 msgr2=0x7f2ae4151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae3e00640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ae40b5b10 0x7f2ae4151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae3e00640 1 -- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ae4072b40 msgr2=0x7f2ae4159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae3e00640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ae4072b40 0x7f2ae4159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae3e00640 1 -- 172.21.15.131:0/1456816126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae415b4d0 con 0x7f2ae4155cd0 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.671+0000 7f2ae3400640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ae4072b40 0x7f2ae4159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.672+0000 7f2ae3e00640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4155cd0 0x7f2ae4161550 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2adc02ad00 tx=0x7f2adc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.672+0000 7f2adb400640 1 -- 172.21.15.131:0/1456816126 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc040070 con 0x7f2ae4155cd0 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.672+0000 7f2adb400640 1 -- 172.21.15.131:0/1456816126 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc031c90 con 0x7f2ae4155cd0 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.672+0000 7f2adb400640 1 -- 172.21.15.131:0/1456816126 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc045680 con 0x7f2ae4155cd0 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.672+0000 7f2ae9000640 1 -- 172.21.15.131:0/1456816126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae415a430 con 0x7f2ae4155cd0 2024-04-28T12:38:47.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.672+0000 7f2ae9000640 1 -- 172.21.15.131:0/1456816126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae415a9f0 con 0x7f2ae4155cd0 2024-04-28T12:38:47.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.673+0000 7f2ae9000640 1 -- 172.21.15.131:0/1456816126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae40506c0 con 0x7f2ae4155cd0 2024-04-28T12:38:47.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.674+0000 7f2adb400640 1 -- 172.21.15.131:0/1456816126 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2adc039040 con 0x7f2ae4155cd0 2024-04-28T12:38:47.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.676+0000 7f2adb400640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2ac406c8f0 0x7f2ac406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:47.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.676+0000 7f2adb400640 1 -- 172.21.15.131:0/1456816126 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f2adc0ad8b0 con 0x7f2ae4155cd0 2024-04-28T12:38:47.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.676+0000 7f2ae3400640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2ac406c8f0 0x7f2ac406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:47.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.677+0000 7f2ae3400640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2ac406c8f0 0x7f2ac406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4006fd0 tx=0x7f2ad4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:47.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.680+0000 7f2adb400640 1 -- 172.21.15.131:0/1456816126 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2adc043360 con 0x7f2ae4155cd0 2024-04-28T12:38:47.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.880+0000 7f2ae9000640 1 -- 172.21.15.131:0/1456816126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2ae40b0fc0 con 0x7f2ae4155cd0 2024-04-28T12:38:47.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.881+0000 7f2adb400640 1 -- 172.21.15.131:0/1456816126 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f2adc07ba30 con 0x7f2ae4155cd0 2024-04-28T12:38:47.878 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:47.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.884+0000 7f2ae9000640 1 -- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2ac406c8f0 msgr2=0x7f2ac406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:47.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.884+0000 7f2ae9000640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2ac406c8f0 0x7f2ac406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4006fd0 tx=0x7f2ad4008040 comp rx=0 tx=0).stop 2024-04-28T12:38:47.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.884+0000 7f2ae9000640 1 -- 172.21.15.131:0/1456816126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4155cd0 msgr2=0x7f2ae4161550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:47.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.884+0000 7f2ae9000640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4155cd0 0x7f2ae4161550 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2adc02ad00 tx=0x7f2adc004a40 comp rx=0 tx=0).stop 2024-04-28T12:38:47.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.884+0000 7f2ae9000640 1 -- 172.21.15.131:0/1456816126 shutdown_connections 2024-04-28T12:38:47.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.884+0000 7f2ae9000640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2ac406c8f0 0x7f2ac406edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.884+0000 7f2ae9000640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ae4155cd0 0x7f2ae4161550 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.884+0000 7f2ae9000640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ae40b5b10 0x7f2ae4151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.884+0000 7f2ae9000640 1 --2- 172.21.15.131:0/1456816126 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ae4072b40 0x7f2ae4159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:47.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.884+0000 7f2ae9000640 1 -- 172.21.15.131:0/1456816126 >> 172.21.15.131:0/1456816126 conn(0x7f2ae4068090 msgr2=0x7f2ae4068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:47.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.884+0000 7f2ae9000640 1 -- 172.21.15.131:0/1456816126 shutdown_connections 2024-04-28T12:38:47.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:47.884+0000 7f2ae9000640 1 -- 172.21.15.131:0/1456816126 wait complete. 2024-04-28T12:38:47.901 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:48.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.104+0000 7f8bec000640 1 Processor -- start 2024-04-28T12:38:48.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.104+0000 7f8bec000640 1 -- start start 2024-04-28T12:38:48.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.105+0000 7f8bec000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8be40b54b0 0x7f8be41604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.105+0000 7f8bec000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8be406df40 0x7f8be415d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.105+0000 7f8bec000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be40b3d80 con 0x7f8be41458e0 2024-04-28T12:38:48.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.105+0000 7f8bec000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f8be40b4060 con 0x7f8be40b54b0 2024-04-28T12:38:48.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.105+0000 7f8bec000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8be40b4340 con 0x7f8be406df40 2024-04-28T12:38:48.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.106+0000 7f8be8c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8be40b54b0 0x7f8be41604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.106+0000 7f8be8c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8be40b54b0 0x7f8be41604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42380/0 (socket says 172.21.15.131:42380) 2024-04-28T12:38:48.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.106+0000 7f8be8c00640 1 -- 172.21.15.131:0/2584282822 learned_addr learned my addr 172.21.15.131:0/2584282822 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:48.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.106+0000 7f8bea000640 1 --2- 172.21.15.131:0/2584282822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8be406df40 0x7f8be415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.106+0000 7f8be8c00640 1 -- 172.21.15.131:0/2584282822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8be406df40 msgr2=0x7f8be415d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:48.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.106+0000 7f8be8c00640 1 --2- 172.21.15.131:0/2584282822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8be406df40 0x7f8be415d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.106+0000 7f8be8c00640 1 -- 172.21.15.131:0/2584282822 >> v1:172.21.15.157:6789/0 conn(0x7f8be41458e0 legacy=0x7f8be40b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:48.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.106+0000 7f8be8c00640 1 -- 172.21.15.131:0/2584282822 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be40b4620 con 0x7f8be40b54b0 2024-04-28T12:38:48.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.107+0000 7f8be8c00640 1 --2- 172.21.15.131:0/2584282822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8be40b54b0 0x7f8be41604d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000bb10 tx=0x7f8bd002c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fec98df0ca995f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:48.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.108+0000 7f8be3e00640 1 -- 172.21.15.131:0/2584282822 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd003a070 con 0x7f8be40b54b0 2024-04-28T12:38:48.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.108+0000 7f8be3e00640 1 -- 172.21.15.131:0/2584282822 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd00326f0 con 0x7f8be40b54b0 2024-04-28T12:38:48.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.108+0000 7f8be3e00640 1 -- 172.21.15.131:0/2584282822 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0032ac0 con 0x7f8be40b54b0 2024-04-28T12:38:48.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.108+0000 7f8bec000640 1 -- 172.21.15.131:0/2584282822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8be40b54b0 msgr2=0x7f8be41604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.108+0000 7f8bec000640 1 --2- 172.21.15.131:0/2584282822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8be40b54b0 0x7f8be41604d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000bb10 tx=0x7f8bd002c830 comp rx=0 tx=0).stop 2024-04-28T12:38:48.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.108+0000 7f8bec000640 1 -- 172.21.15.131:0/2584282822 shutdown_connections 2024-04-28T12:38:48.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.108+0000 7f8bec000640 1 --2- 172.21.15.131:0/2584282822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8be406df40 0x7f8be415d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.108+0000 7f8bec000640 1 --2- 172.21.15.131:0/2584282822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8be40b54b0 0x7f8be41604d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.108+0000 7f8bec000640 1 -- 172.21.15.131:0/2584282822 >> 172.21.15.131:0/2584282822 conn(0x7f8be40b2900 msgr2=0x7f8be40b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:48.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.109+0000 7f8bec000640 1 -- 172.21.15.131:0/2584282822 shutdown_connections 2024-04-28T12:38:48.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.109+0000 7f8bec000640 1 -- 172.21.15.131:0/2584282822 wait complete. 2024-04-28T12:38:48.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.110+0000 7f8bec000640 1 Processor -- start 2024-04-28T12:38:48.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.110+0000 7f8bec000640 1 -- start start 2024-04-28T12:38:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.111+0000 7f8bec000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8be406df40 0x7f8be4139690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.111+0000 7f8bec000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be40b54b0 0x7f8be40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.111+0000 7f8bec000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8be41458e0 0x7f8be40d0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.111+0000 7f8bec000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8be415ddc0 con 0x7f8be406df40 2024-04-28T12:38:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.111+0000 7f8bec000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8be415dc40 con 0x7f8be40b54b0 2024-04-28T12:38:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.111+0000 7f8bec000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8be415df40 con 0x7f8be41458e0 2024-04-28T12:38:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.112+0000 7f8be9600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8be406df40 0x7f8be4139690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.112+0000 7f8be9600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8be406df40 0x7f8be4139690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42388/0 (socket says 172.21.15.131:42388) 2024-04-28T12:38:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.112+0000 7f8be9600640 1 -- 172.21.15.131:0/1950398421 learned_addr learned my addr 172.21.15.131:0/1950398421 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.112+0000 7f8be8c00640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be40b54b0 0x7f8be40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.112+0000 7f8be9600640 1 -- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8be41458e0 msgr2=0x7f8be40d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.112+0000 7f8bea000640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8be41458e0 0x7f8be40d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.112+0000 7f8be9600640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8be41458e0 0x7f8be40d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.112+0000 7f8be9600640 1 -- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be40b54b0 msgr2=0x7f8be40cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.112+0000 7f8be9600640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be40b54b0 0x7f8be40cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.112+0000 7f8be9600640 1 -- 172.21.15.131:0/1950398421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be40ce680 con 0x7f8be406df40 2024-04-28T12:38:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.112+0000 7f8be8c00640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be40b54b0 0x7f8be40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:48.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.112+0000 7f8be9600640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8be406df40 0x7f8be4139690 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800b410 tx=0x7f8bd800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.113+0000 7f8be1600640 1 -- 172.21.15.131:0/1950398421 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8015020 con 0x7f8be406df40 2024-04-28T12:38:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.113+0000 7f8be1600640 1 -- 172.21.15.131:0/1950398421 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd8011830 con 0x7f8be406df40 2024-04-28T12:38:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.113+0000 7f8be1600640 1 -- 172.21.15.131:0/1950398421 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8011c20 con 0x7f8be406df40 2024-04-28T12:38:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.113+0000 7f8bec000640 1 -- 172.21.15.131:0/1950398421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be40ce960 con 0x7f8be406df40 2024-04-28T12:38:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.113+0000 7f8bec000640 1 -- 172.21.15.131:0/1950398421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be40cef20 con 0x7f8be406df40 2024-04-28T12:38:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.114+0000 7f8bec000640 1 -- 172.21.15.131:0/1950398421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be4069350 con 0x7f8be406df40 2024-04-28T12:38:48.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.115+0000 7f8be1600640 1 -- 172.21.15.131:0/1950398421 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bd8018020 con 0x7f8be406df40 2024-04-28T12:38:48.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.116+0000 7f8be1600640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8bc406c920 0x7f8bc406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.116+0000 7f8be1600640 1 -- 172.21.15.131:0/1950398421 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8bd808ba00 con 0x7f8be406df40 2024-04-28T12:38:48.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.117+0000 7f8be8c00640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8bc406c920 0x7f8bc406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.118+0000 7f8be8c00640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8bc406c920 0x7f8bc406ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0037000 tx=0x7f8bd0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.120+0000 7f8be1600640 1 -- 172.21.15.131:0/1950398421 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8bd801f070 con 0x7f8be406df40 2024-04-28T12:38:48.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.310+0000 7f8bec000640 1 -- 172.21.15.131:0/1950398421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8be406e5b0 con 0x7f8be406df40 2024-04-28T12:38:48.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.311+0000 7f8be1600640 1 -- 172.21.15.131:0/1950398421 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f8bd8018310 con 0x7f8be406df40 2024-04-28T12:38:48.308 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:48.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.314+0000 7f8bec000640 1 -- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8bc406c920 msgr2=0x7f8bc406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.314+0000 7f8bec000640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8bc406c920 0x7f8bc406ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0037000 tx=0x7f8bd0008040 comp rx=0 tx=0).stop 2024-04-28T12:38:48.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.314+0000 7f8bec000640 1 -- 172.21.15.131:0/1950398421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8be406df40 msgr2=0x7f8be4139690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.314+0000 7f8bec000640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8be406df40 0x7f8be4139690 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800b410 tx=0x7f8bd800b8d0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.314+0000 7f8bec000640 1 -- 172.21.15.131:0/1950398421 shutdown_connections 2024-04-28T12:38:48.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.314+0000 7f8bec000640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8bc406c920 0x7f8bc406ede0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.314+0000 7f8bec000640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8be41458e0 0x7f8be40d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.314+0000 7f8bec000640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8be40b54b0 0x7f8be40cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.314+0000 7f8bec000640 1 --2- 172.21.15.131:0/1950398421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8be406df40 0x7f8be4139690 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.314+0000 7f8bec000640 1 -- 172.21.15.131:0/1950398421 >> 172.21.15.131:0/1950398421 conn(0x7f8be40b2900 msgr2=0x7f8be40aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:48.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.314+0000 7f8bec000640 1 -- 172.21.15.131:0/1950398421 shutdown_connections 2024-04-28T12:38:48.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.314+0000 7f8bec000640 1 -- 172.21.15.131:0/1950398421 wait complete. 2024-04-28T12:38:48.328 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.531+0000 7f28f3e00640 1 Processor -- start 2024-04-28T12:38:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.532+0000 7f28f3e00640 1 -- start start 2024-04-28T12:38:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.533+0000 7f28f3e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f28f40d38e0 0x7f28f40d5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.533+0000 7f28f3e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f40cfa30 con 0x7f28f41636a0 2024-04-28T12:38:48.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.533+0000 7f28f3e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f40cfd10 con 0x7f28f41624f0 2024-04-28T12:38:48.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.533+0000 7f28f3e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f28f40cf800 con 0x7f28f40d38e0 2024-04-28T12:38:48.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.533+0000 7f28f3400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f28f41636a0 0x7f28f40b17c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49396/0 (socket says 172.21.15.131:49396) 2024-04-28T12:38:48.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.533+0000 7f28f3400640 1 -- 172.21.15.131:0/456893086 learned_addr learned my addr 172.21.15.131:0/456893086 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f2000640 1 --2- 172.21.15.131:0/456893086 >> v2:172.21.15.157:3300/0 conn(0x7f28f40d38e0 0x7f28f40d5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 504184423 0 0) 0x7f28f40cfa30 con 0x7f28f41636a0 2024-04-28T12:38:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28d8003520 con 0x7f28f41636a0 2024-04-28T12:38:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 963677269 0 0) 0x7f28d8003520 con 0x7f28f41636a0 2024-04-28T12:38:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 >> v2:172.21.15.157:3300/0 conn(0x7f28f40d38e0 msgr2=0x7f28f40d5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 --2- 172.21.15.131:0/456893086 >> v2:172.21.15.157:3300/0 conn(0x7f28f40d38e0 0x7f28f40d5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 >> v1:172.21.15.157:6789/0 conn(0x7f28f41624f0 legacy=0x7f28f40d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f40cfff0 con 0x7f28f41636a0 2024-04-28T12:38:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f28ec0019f0 con 0x7f28f41636a0 2024-04-28T12:38:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 >> v1:172.21.15.131:6789/0 conn(0x7f28f41636a0 legacy=0x7f28f40b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 --2- 172.21.15.131:0/456893086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28d8003a70 0x7f28d8003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 --2- 172.21.15.131:0/456893086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28d8004800 0x7f28d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 --2- 172.21.15.131:0/456893086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d8005210 0x7f28d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.534+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f28d800c120 con 0x7f28d8003a70 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.535+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f28d800c2a0 con 0x7f28d8004800 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.535+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f28d800c4a0 con 0x7f28d8005210 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.535+0000 7f28f2a00640 1 --2- 172.21.15.131:0/456893086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28d8003a70 0x7f28d8003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.535+0000 7f28f2000640 1 --2- 172.21.15.131:0/456893086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28d8004800 0x7f28d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.535+0000 7f28f2a00640 1 -- 172.21.15.131:0/456893086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d8005210 msgr2=0x7f28d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.535+0000 7f28f2a00640 1 --2- 172.21.15.131:0/456893086 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d8005210 0x7f28d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.535+0000 7f28f2a00640 1 -- 172.21.15.131:0/456893086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28d8004800 msgr2=0x7f28d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.535+0000 7f28f2a00640 1 --2- 172.21.15.131:0/456893086 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28d8004800 0x7f28d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.535+0000 7f28f2a00640 1 -- 172.21.15.131:0/456893086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d8003520 con 0x7f28d8003a70 2024-04-28T12:38:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.535+0000 7f28f3400640 1 -- 172.21.15.131:0/456893086 reap_dead start 2024-04-28T12:38:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.536+0000 7f28f2a00640 1 --2- 172.21.15.131:0/456893086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28d8003a70 0x7f28d8003e90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f28e4001750 tx=0x7f28d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29efb895cac08a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.536+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4001370 con 0x7f28d8003a70 2024-04-28T12:38:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.536+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e40052d0 con 0x7f28d8003a70 2024-04-28T12:38:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.536+0000 7f28f1600640 1 -- 172.21.15.131:0/456893086 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e402abc0 con 0x7f28d8003a70 2024-04-28T12:38:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.537+0000 7f28f3e00640 1 -- 172.21.15.131:0/456893086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28d8003a70 msgr2=0x7f28d8003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.537+0000 7f28f3e00640 1 --2- 172.21.15.131:0/456893086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28d8003a70 0x7f28d8003e90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f28e4001750 tx=0x7f28d8005830 comp rx=0 tx=0).stop 2024-04-28T12:38:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.537+0000 7f28f3e00640 1 -- 172.21.15.131:0/456893086 shutdown_connections 2024-04-28T12:38:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.537+0000 7f28f3e00640 1 --2- 172.21.15.131:0/456893086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28d8003a70 0x7f28d8003e90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.537+0000 7f28f3e00640 1 -- 172.21.15.131:0/456893086 >> 172.21.15.131:0/456893086 conn(0x7f28f40d4bf0 msgr2=0x7f28f40d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.537+0000 7f28f3e00640 1 -- 172.21.15.131:0/456893086 shutdown_connections 2024-04-28T12:38:48.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.537+0000 7f28f3e00640 1 -- 172.21.15.131:0/456893086 wait complete. 2024-04-28T12:38:48.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.539+0000 7f28f3e00640 1 Processor -- start 2024-04-28T12:38:48.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.539+0000 7f28f3e00640 1 -- start start 2024-04-28T12:38:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.540+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d8003a70 0x7f28f40cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.540+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f41f59b0 0x7f28f40b9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.540+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28f409fb80 0x7f28f40a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.540+0000 7f28f3e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f28f40d0360 con 0x7f28f409fb80 2024-04-28T12:38:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.540+0000 7f28f3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4163bf0 con 0x7f28f41f59b0 2024-04-28T12:38:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.540+0000 7f28f3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f28f41628d0 con 0x7f28d8003a70 2024-04-28T12:38:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.540+0000 7f28f3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28f409fb80 0x7f28f40a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.540+0000 7f28f3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28f409fb80 0x7f28f40a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42416/0 (socket says 172.21.15.131:42416) 2024-04-28T12:38:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.540+0000 7f28f3400640 1 -- 172.21.15.131:0/1883855366 learned_addr learned my addr 172.21.15.131:0/1883855366 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:48.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.541+0000 7f28f2a00640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d8003a70 0x7f28f40cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.541+0000 7f28f2000640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f41f59b0 0x7f28f40b9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.541+0000 7f28f3400640 1 -- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d8003a70 msgr2=0x7f28f40cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.541+0000 7f28f3400640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d8003a70 0x7f28f40cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.541+0000 7f28f3400640 1 -- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f41f59b0 msgr2=0x7f28f40b9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.541+0000 7f28f3400640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f41f59b0 0x7f28f40b9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.541+0000 7f28f3400640 1 -- 172.21.15.131:0/1883855366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f40ba510 con 0x7f28f409fb80 2024-04-28T12:38:48.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.541+0000 7f28f3400640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28f409fb80 0x7f28f40a0570 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f28f408f140 tx=0x7f28ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:48.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.541+0000 7f28eaa00640 1 -- 172.21.15.131:0/1883855366 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec00a5b0 con 0x7f28f409fb80 2024-04-28T12:38:48.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.541+0000 7f28eaa00640 1 -- 172.21.15.131:0/1883855366 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ec00a770 con 0x7f28f409fb80 2024-04-28T12:38:48.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.541+0000 7f28eaa00640 1 -- 172.21.15.131:0/1883855366 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec00ab60 con 0x7f28f409fb80 2024-04-28T12:38:48.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.541+0000 7f28f3e00640 1 -- 172.21.15.131:0/1883855366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f409eae0 con 0x7f28f409fb80 2024-04-28T12:38:48.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.542+0000 7f28f3e00640 1 -- 172.21.15.131:0/1883855366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f409f0a0 con 0x7f28f409fb80 2024-04-28T12:38:48.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.543+0000 7f28f3e00640 1 -- 172.21.15.131:0/1883855366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f40506c0 con 0x7f28f409fb80 2024-04-28T12:38:48.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.544+0000 7f28eaa00640 1 -- 172.21.15.131:0/1883855366 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28ec011070 con 0x7f28f409fb80 2024-04-28T12:38:48.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.545+0000 7f28eaa00640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f28d006c920 0x7f28d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.545+0000 7f28eaa00640 1 -- 172.21.15.131:0/1883855366 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f28ec08b580 con 0x7f28f409fb80 2024-04-28T12:38:48.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.546+0000 7f28f2a00640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f28d006c920 0x7f28d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.547+0000 7f28f2a00640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f28d006c920 0x7f28d006ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f28e4032000 tx=0x7f28e402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:48.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.549+0000 7f28eaa00640 1 -- 172.21.15.131:0/1883855366 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28ec059510 con 0x7f28f409fb80 2024-04-28T12:38:48.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.748+0000 7f28f3e00640 1 -- 172.21.15.131:0/1883855366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f28f408ed10 con 0x7f28f409fb80 2024-04-28T12:38:48.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.749+0000 7f28eaa00640 1 -- 172.21.15.131:0/1883855366 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f28ec058e10 con 0x7f28f409fb80 2024-04-28T12:38:48.745 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.751+0000 7f28f3e00640 1 -- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f28d006c920 msgr2=0x7f28d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.751+0000 7f28f3e00640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f28d006c920 0x7f28d006ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f28e4032000 tx=0x7f28e402e000 comp rx=0 tx=0).stop 2024-04-28T12:38:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.751+0000 7f28f3e00640 1 -- 172.21.15.131:0/1883855366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28f409fb80 msgr2=0x7f28f40a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.751+0000 7f28f3e00640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28f409fb80 0x7f28f40a0570 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f28f408f140 tx=0x7f28ec00cd50 comp rx=0 tx=0).stop 2024-04-28T12:38:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.752+0000 7f28f3e00640 1 -- 172.21.15.131:0/1883855366 shutdown_connections 2024-04-28T12:38:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.752+0000 7f28f3e00640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f28d006c920 0x7f28d006ede0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.752+0000 7f28f3e00640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28f409fb80 0x7f28f40a0570 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.752+0000 7f28f3e00640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28f41f59b0 0x7f28f40b9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.752+0000 7f28f3e00640 1 --2- 172.21.15.131:0/1883855366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d8003a70 0x7f28f40cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.752+0000 7f28f3e00640 1 -- 172.21.15.131:0/1883855366 >> 172.21.15.131:0/1883855366 conn(0x7f28f40d4bf0 msgr2=0x7f28f406dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.752+0000 7f28f3e00640 1 -- 172.21.15.131:0/1883855366 shutdown_connections 2024-04-28T12:38:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.752+0000 7f28f3e00640 1 -- 172.21.15.131:0/1883855366 wait complete. 2024-04-28T12:38:48.768 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:48.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.977+0000 7f72c2400640 1 Processor -- start 2024-04-28T12:38:48.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.977+0000 7f72c2400640 1 -- start start 2024-04-28T12:38:48.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.978+0000 7f72c2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72bc15a7e0 0x7f72bc0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.978+0000 7f72c2400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72bc0b3900 con 0x7f72bc15d9c0 2024-04-28T12:38:48.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.978+0000 7f72c2400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72bc0b3b70 con 0x7f72bc0b5410 2024-04-28T12:38:48.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.978+0000 7f72c2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f72bc15abc0 con 0x7f72bc15a7e0 2024-04-28T12:38:48.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.979+0000 7f72c1a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f72bc15d9c0 0x7f72bc15e2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49406/0 (socket says 172.21.15.131:49406) 2024-04-28T12:38:48.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.979+0000 7f72c1a00640 1 -- 172.21.15.131:0/4199971667 learned_addr learned my addr 172.21.15.131:0/4199971667 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.979+0000 7f72c1000640 1 --2- 172.21.15.131:0/4199971667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72bc15a7e0 0x7f72bc0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.979+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 922949747 0 0) 0x7f72bc0b3900 con 0x7f72bc15d9c0 2024-04-28T12:38:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.979+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72a4003520 con 0x7f72bc15d9c0 2024-04-28T12:38:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.979+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1160208230 0 0) 0x7f72a4003520 con 0x7f72bc15d9c0 2024-04-28T12:38:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.979+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72bc15a7e0 msgr2=0x7f72bc0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.979+0000 7f72bb400640 1 --2- 172.21.15.131:0/4199971667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72bc15a7e0 0x7f72bc0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 >> v1:172.21.15.157:6789/0 conn(0x7f72bc0b5410 legacy=0x7f72bc15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc0b3e50 con 0x7f72bc15d9c0 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f72b40019f0 con 0x7f72bc15d9c0 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 >> v1:172.21.15.131:6789/0 conn(0x7f72bc15d9c0 legacy=0x7f72bc15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72bb400640 1 --2- 172.21.15.131:0/4199971667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a4003a50 0x7f72a4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72bb400640 1 --2- 172.21.15.131:0/4199971667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a4004800 0x7f72a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72bb400640 1 --2- 172.21.15.131:0/4199971667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72bc15a7e0 0x7f72a4005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc15abc0 con 0x7f72a4003a50 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f72a400bb80 con 0x7f72a4004800 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f72a400bd80 con 0x7f72bc15a7e0 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72c1000640 1 --2- 172.21.15.131:0/4199971667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a4003a50 0x7f72a4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72c1000640 1 -- 172.21.15.131:0/4199971667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72bc15a7e0 msgr2=0x7f72a4005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:48.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72c1000640 1 --2- 172.21.15.131:0/4199971667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72bc15a7e0 0x7f72a4005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72c1000640 1 -- 172.21.15.131:0/4199971667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a4004800 msgr2=0x7f72a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:48.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72c1000640 1 --2- 172.21.15.131:0/4199971667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a4004800 0x7f72a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.980+0000 7f72c1000640 1 -- 172.21.15.131:0/4199971667 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a4003520 con 0x7f72a4003a50 2024-04-28T12:38:48.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.981+0000 7f72c1000640 1 --2- 172.21.15.131:0/4199971667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a4003a50 0x7f72a4003e70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f72b000ba30 tx=0x7f72a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f01e594ef07fd87 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:48.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.982+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0030070 con 0x7f72a4003a50 2024-04-28T12:38:48.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.982+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72b002ae10 con 0x7f72a4003a50 2024-04-28T12:38:48.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.982+0000 7f72bb400640 1 -- 172.21.15.131:0/4199971667 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b000b600 con 0x7f72a4003a50 2024-04-28T12:38:48.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.982+0000 7f72c2400640 1 -- 172.21.15.131:0/4199971667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a4003a50 msgr2=0x7f72a4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.982+0000 7f72c2400640 1 --2- 172.21.15.131:0/4199971667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72a4003a50 0x7f72a4003e70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f72b000ba30 tx=0x7f72a4005830 comp rx=0 tx=0).stop 2024-04-28T12:38:48.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.982+0000 7f72c1a00640 1 -- 172.21.15.131:0/4199971667 reap_dead start 2024-04-28T12:38:48.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.982+0000 7f72c2400640 1 -- 172.21.15.131:0/4199971667 shutdown_connections 2024-04-28T12:38:48.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.982+0000 7f72c2400640 1 -- 172.21.15.131:0/4199971667 >> 172.21.15.131:0/4199971667 conn(0x7f72bc068090 msgr2=0x7f72bc0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:48.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.982+0000 7f72c2400640 1 -- 172.21.15.131:0/4199971667 shutdown_connections 2024-04-28T12:38:48.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.982+0000 7f72c2400640 1 -- 172.21.15.131:0/4199971667 wait complete. 2024-04-28T12:38:48.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.984+0000 7f72c2400640 1 Processor -- start 2024-04-28T12:38:48.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.984+0000 7f72c2400640 1 -- start start 2024-04-28T12:38:48.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.985+0000 7f72c2400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a4003a50 0x7f72bc158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.985+0000 7f72c2400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72bc159400 0x7f72bc1e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.985+0000 7f72c2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72bc1e7440 0x7f72bc0ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.985+0000 7f72c2400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc15e9f0 con 0x7f72bc159400 2024-04-28T12:38:48.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.985+0000 7f72c2400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc15e010 con 0x7f72a4003a50 2024-04-28T12:38:48.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.985+0000 7f72c2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f72bc15eb70 con 0x7f72bc1e7440 2024-04-28T12:38:48.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.985+0000 7f72bbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72bc159400 0x7f72bc1e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.985+0000 7f72bbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72bc159400 0x7f72bc1e6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42440/0 (socket says 172.21.15.131:42440) 2024-04-28T12:38:48.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.985+0000 7f72bbe00640 1 -- 172.21.15.131:0/1355329578 learned_addr learned my addr 172.21.15.131:0/1355329578 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.986+0000 7f72c1000640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a4003a50 0x7f72bc158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.986+0000 7f72c1a00640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72bc1e7440 0x7f72bc0ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.986+0000 7f72bbe00640 1 -- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72bc1e7440 msgr2=0x7f72bc0ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.986+0000 7f72bbe00640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72bc1e7440 0x7f72bc0ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.986+0000 7f72bbe00640 1 -- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a4003a50 msgr2=0x7f72bc158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.986+0000 7f72bbe00640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a4003a50 0x7f72bc158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.986+0000 7f72bbe00640 1 -- 172.21.15.131:0/1355329578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc1e78e0 con 0x7f72bc159400 2024-04-28T12:38:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.986+0000 7f72bbe00640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72bc159400 0x7f72bc1e6f00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f72a800c500 tx=0x7f72a800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.986+0000 7f72b8c00640 1 -- 172.21.15.131:0/1355329578 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a80052d0 con 0x7f72bc159400 2024-04-28T12:38:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.986+0000 7f72b8c00640 1 -- 172.21.15.131:0/1355329578 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a8005470 con 0x7f72bc159400 2024-04-28T12:38:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.986+0000 7f72b8c00640 1 -- 172.21.15.131:0/1355329578 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8012870 con 0x7f72bc159400 2024-04-28T12:38:48.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.986+0000 7f72c2400640 1 -- 172.21.15.131:0/1355329578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72bc071db0 con 0x7f72bc159400 2024-04-28T12:38:48.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.987+0000 7f72c2400640 1 -- 172.21.15.131:0/1355329578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72bc158ab0 con 0x7f72bc159400 2024-04-28T12:38:48.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.988+0000 7f72c2400640 1 -- 172.21.15.131:0/1355329578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72bc0506c0 con 0x7f72bc159400 2024-04-28T12:38:48.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.989+0000 7f72b8c00640 1 -- 172.21.15.131:0/1355329578 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72a8012a10 con 0x7f72bc159400 2024-04-28T12:38:48.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.990+0000 7f72b8c00640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f729c06c8f0 0x7f729c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:48.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.990+0000 7f72b8c00640 1 -- 172.21.15.131:0/1355329578 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f72a808bc70 con 0x7f72bc159400 2024-04-28T12:38:48.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.990+0000 7f72c1000640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f729c06c8f0 0x7f729c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:48.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.991+0000 7f72c1000640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f729c06c8f0 0x7f729c06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f72b000ba00 tx=0x7f72b0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:48.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:48.994+0000 7f72b8c00640 1 -- 172.21.15.131:0/1355329578 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72a8059c90 con 0x7f72bc159400 2024-04-28T12:38:49.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.191+0000 7f72c2400640 1 -- 172.21.15.131:0/1355329578 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f72bc071b60 con 0x7f72bc159400 2024-04-28T12:38:49.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.192+0000 7f72b8c00640 1 -- 172.21.15.131:0/1355329578 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f72a8059590 con 0x7f72bc159400 2024-04-28T12:38:49.189 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:49.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.194+0000 7f72c2400640 1 -- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f729c06c8f0 msgr2=0x7f729c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.194+0000 7f72c2400640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f729c06c8f0 0x7f729c06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f72b000ba00 tx=0x7f72b0030000 comp rx=0 tx=0).stop 2024-04-28T12:38:49.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.194+0000 7f72c2400640 1 -- 172.21.15.131:0/1355329578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72bc159400 msgr2=0x7f72bc1e6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.194+0000 7f72c2400640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72bc159400 0x7f72bc1e6f00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f72a800c500 tx=0x7f72a800c9c0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.195+0000 7f72c2400640 1 -- 172.21.15.131:0/1355329578 shutdown_connections 2024-04-28T12:38:49.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.195+0000 7f72c2400640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f729c06c8f0 0x7f729c06edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.195+0000 7f72c2400640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72bc1e7440 0x7f72bc0ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.195+0000 7f72c2400640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72bc159400 0x7f72bc1e6f00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.195+0000 7f72c2400640 1 --2- 172.21.15.131:0/1355329578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72a4003a50 0x7f72bc158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.195+0000 7f72c2400640 1 -- 172.21.15.131:0/1355329578 >> 172.21.15.131:0/1355329578 conn(0x7f72bc068090 msgr2=0x7f72bc0b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:49.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.195+0000 7f72c2400640 1 -- 172.21.15.131:0/1355329578 shutdown_connections 2024-04-28T12:38:49.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.195+0000 7f72c2400640 1 -- 172.21.15.131:0/1355329578 wait complete. 2024-04-28T12:38:49.211 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T12:38:49.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.416+0000 7f0162e00640 1 Processor -- start 2024-04-28T12:38:49.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.417+0000 7f0162e00640 1 -- start start 2024-04-28T12:38:49.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.418+0000 7f0162e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f015c0b3490 0x7f015c06f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.418+0000 7f0162e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f015c0cf060 con 0x7f015c06f560 2024-04-28T12:38:49.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.418+0000 7f0162e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f015c0cf340 con 0x7f015c145590 2024-04-28T12:38:49.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.418+0000 7f0162e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f015c0d0100 con 0x7f015c0b3490 2024-04-28T12:38:49.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.418+0000 7f0160e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f015c06f560 0x7f015c071eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49418/0 (socket says 172.21.15.131:49418) 2024-04-28T12:38:49.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.418+0000 7f0160e00640 1 -- 172.21.15.131:0/2521971305 learned_addr learned my addr 172.21.15.131:0/2521971305 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:49.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015b400640 1 --2- 172.21.15.131:0/2521971305 >> v2:172.21.15.157:3300/0 conn(0x7f015c0b3490 0x7f015c06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:49.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670917214 0 0) 0x7f015c0cf060 con 0x7f015c06f560 2024-04-28T12:38:49.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0140003520 con 0x7f015c06f560 2024-04-28T12:38:49.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 751499619 0 0) 0x7f0140003520 con 0x7f015c06f560 2024-04-28T12:38:49.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 >> v2:172.21.15.157:3300/0 conn(0x7f015c0b3490 msgr2=0x7f015c06f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:49.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 --2- 172.21.15.131:0/2521971305 >> v2:172.21.15.157:3300/0 conn(0x7f015c0b3490 0x7f015c06f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 >> v1:172.21.15.157:6789/0 conn(0x7f015c145590 legacy=0x7f015c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c0cf620 con 0x7f015c06f560 2024-04-28T12:38:49.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f0154001670 con 0x7f015c06f560 2024-04-28T12:38:49.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 >> v1:172.21.15.131:6789/0 conn(0x7f015c06f560 legacy=0x7f015c071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 --2- 172.21.15.131:0/2521971305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01400039b0 0x7f0140003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 --2- 172.21.15.131:0/2521971305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0140004800 0x7f0140004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 --2- 172.21.15.131:0/2521971305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01400051c0 0x7f014000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f015c0d0100 con 0x7f01400039b0 2024-04-28T12:38:49.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f014000c060 con 0x7f01400051c0 2024-04-28T12:38:49.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f014000c260 con 0x7f0140004800 2024-04-28T12:38:49.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.419+0000 7f015be00640 1 --2- 172.21.15.131:0/2521971305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01400039b0 0x7f0140003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:49.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.420+0000 7f015be00640 1 -- 172.21.15.131:0/2521971305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0140004800 msgr2=0x7f0140004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:49.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.420+0000 7f015be00640 1 --2- 172.21.15.131:0/2521971305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0140004800 0x7f0140004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.420+0000 7f015be00640 1 -- 172.21.15.131:0/2521971305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01400051c0 msgr2=0x7f014000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:49.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.420+0000 7f015be00640 1 --2- 172.21.15.131:0/2521971305 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01400051c0 0x7f014000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.420+0000 7f015be00640 1 -- 172.21.15.131:0/2521971305 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0140003520 con 0x7f01400039b0 2024-04-28T12:38:49.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.420+0000 7f0160e00640 1 -- 172.21.15.131:0/2521971305 reap_dead start 2024-04-28T12:38:49.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.421+0000 7f015be00640 1 --2- 172.21.15.131:0/2521971305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01400039b0 0x7f0140003dd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f014c001e50 tx=0x7f0140005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b074c8e6c8182b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:49.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.421+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c001a70 con 0x7f01400039b0 2024-04-28T12:38:49.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.421+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f014c003970 con 0x7f01400039b0 2024-04-28T12:38:49.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.421+0000 7f015aa00640 1 -- 172.21.15.131:0/2521971305 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c003d60 con 0x7f01400039b0 2024-04-28T12:38:49.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.422+0000 7f0162e00640 1 -- 172.21.15.131:0/2521971305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01400039b0 msgr2=0x7f0140003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.422+0000 7f0162e00640 1 --2- 172.21.15.131:0/2521971305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01400039b0 0x7f0140003dd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f014c001e50 tx=0x7f0140005830 comp rx=0 tx=0).stop 2024-04-28T12:38:49.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.422+0000 7f0162e00640 1 -- 172.21.15.131:0/2521971305 shutdown_connections 2024-04-28T12:38:49.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.422+0000 7f0162e00640 1 --2- 172.21.15.131:0/2521971305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01400039b0 0x7f0140003dd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.422+0000 7f0162e00640 1 -- 172.21.15.131:0/2521971305 >> 172.21.15.131:0/2521971305 conn(0x7f015c0b2900 msgr2=0x7f015c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:49.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.422+0000 7f0162e00640 1 -- 172.21.15.131:0/2521971305 shutdown_connections 2024-04-28T12:38:49.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.422+0000 7f0162e00640 1 -- 172.21.15.131:0/2521971305 wait complete. 2024-04-28T12:38:49.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.424+0000 7f0162e00640 1 Processor -- start 2024-04-28T12:38:49.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.424+0000 7f0162e00640 1 -- start start 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0162e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01400039b0 0x7f015c1e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0162e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c13df50 0x7f015c1f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0162e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f015c13c8f0 0x7f015c1e8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0162e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f015c0cfbe0 con 0x7f015c13c8f0 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0162e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f015c06f990 con 0x7f01400039b0 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0162e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f015c145970 con 0x7f015c13df50 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0160e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f015c13c8f0 0x7f015c1e8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0160e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f015c13c8f0 0x7f015c1e8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42460/0 (socket says 172.21.15.131:42460) 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0160e00640 1 -- 172.21.15.131:0/4085971834 learned_addr learned my addr 172.21.15.131:0/4085971834 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f015b400640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c13df50 0x7f015c1f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f015be00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01400039b0 0x7f015c1e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0160e00640 1 -- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c13df50 msgr2=0x7f015c1f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0160e00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c13df50 0x7f015c1f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0160e00640 1 -- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01400039b0 msgr2=0x7f015c1e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0160e00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01400039b0 0x7f015c1e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f0160e00640 1 -- 172.21.15.131:0/4085971834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c13ceb0 con 0x7f015c13c8f0 2024-04-28T12:38:49.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.425+0000 7f015be00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01400039b0 0x7f015c1e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:49.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.426+0000 7f0160e00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f015c13c8f0 0x7f015c1e8670 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0154001050 tx=0x7f015400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:49.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.426+0000 7f0153e00640 1 -- 172.21.15.131:0/4085971834 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01540074e0 con 0x7f015c13c8f0 2024-04-28T12:38:49.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.426+0000 7f0153e00640 1 -- 172.21.15.131:0/4085971834 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01540047e0 con 0x7f015c13c8f0 2024-04-28T12:38:49.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.426+0000 7f0162e00640 1 -- 172.21.15.131:0/4085971834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015c13d190 con 0x7f015c13c8f0 2024-04-28T12:38:49.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.426+0000 7f0153e00640 1 -- 172.21.15.131:0/4085971834 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154004bd0 con 0x7f015c13c8f0 2024-04-28T12:38:49.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.426+0000 7f0162e00640 1 -- 172.21.15.131:0/4085971834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015c13d750 con 0x7f015c13c8f0 2024-04-28T12:38:49.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.428+0000 7f0162e00640 1 -- 172.21.15.131:0/4085971834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015c069350 con 0x7f015c13c8f0 2024-04-28T12:38:49.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.428+0000 7f0153e00640 1 -- 172.21.15.131:0/4085971834 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f015401c3f0 con 0x7f015c13c8f0 2024-04-28T12:38:49.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.430+0000 7f0153e00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f013c06c920 0x7f013c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.430+0000 7f0153e00640 1 -- 172.21.15.131:0/4085971834 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f015408b670 con 0x7f015c13c8f0 2024-04-28T12:38:49.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.430+0000 7f015be00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f013c06c920 0x7f013c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:49.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.432+0000 7f015be00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f013c06c920 0x7f013c06ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f014c032000 tx=0x7f014c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:49.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.435+0000 7f0153e00640 1 -- 172.21.15.131:0/4085971834 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f01540596a0 con 0x7f015c13c8f0 2024-04-28T12:38:49.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.645+0000 7f0162e00640 1 -- 172.21.15.131:0/4085971834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f015c144390 con 0x7f015c13c8f0 2024-04-28T12:38:49.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.646+0000 7f0153e00640 1 -- 172.21.15.131:0/4085971834 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f0154058fa0 con 0x7f015c13c8f0 2024-04-28T12:38:49.642 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-04-28T12:38:49.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.648+0000 7f0162e00640 1 -- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f013c06c920 msgr2=0x7f013c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.648+0000 7f0162e00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f013c06c920 0x7f013c06ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f014c032000 tx=0x7f014c02e000 comp rx=0 tx=0).stop 2024-04-28T12:38:49.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.649+0000 7f0162e00640 1 -- 172.21.15.131:0/4085971834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f015c13c8f0 msgr2=0x7f015c1e8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.649+0000 7f0162e00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f015c13c8f0 0x7f015c1e8670 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0154001050 tx=0x7f015400c890 comp rx=0 tx=0).stop 2024-04-28T12:38:49.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.649+0000 7f0162e00640 1 -- 172.21.15.131:0/4085971834 shutdown_connections 2024-04-28T12:38:49.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.649+0000 7f0162e00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f013c06c920 0x7f013c06ede0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.649+0000 7f0162e00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f015c13c8f0 0x7f015c1e8670 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.649+0000 7f0162e00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f015c13df50 0x7f015c1f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.649+0000 7f0162e00640 1 --2- 172.21.15.131:0/4085971834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01400039b0 0x7f015c1e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.649+0000 7f0162e00640 1 -- 172.21.15.131:0/4085971834 >> 172.21.15.131:0/4085971834 conn(0x7f015c0b2900 msgr2=0x7f015c13c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:49.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.649+0000 7f0162e00640 1 -- 172.21.15.131:0/4085971834 shutdown_connections 2024-04-28T12:38:49.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.649+0000 7f0162e00640 1 -- 172.21.15.131:0/4085971834 wait complete. 2024-04-28T12:38:49.667 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:38:49.667 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:38:49.725 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:49.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.931+0000 7f1e53600640 1 Processor -- start 2024-04-28T12:38:49.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.931+0000 7f1e53600640 1 -- start start 2024-04-28T12:38:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.932+0000 7f1e53600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1e4c15cab0 0x7f1e4c0afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.933+0000 7f1e53600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f1e4c0d1250 0x7f1e4c15f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.933+0000 7f1e53600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e4c0b0650 con 0x7f1e4c0b0020 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.933+0000 7f1e53600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1e4c0d0900 con 0x7f1e4c15cab0 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.933+0000 7f1e53600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1e4c1635a0 con 0x7f1e4c0d1250 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.933+0000 7f1e50c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1e4c15cab0 0x7f1e4c0afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.933+0000 7f1e4be00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1e4c0b0020 0x7f1e4c0d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49420/0 (socket says 172.21.15.131:49420) 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.933+0000 7f1e4be00640 1 -- 172.21.15.131:0/2249836018 learned_addr learned my addr 172.21.15.131:0/2249836018 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.933+0000 7f1e50c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1e4c15cab0 0x7f1e4c0afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42468/0 (socket says 172.21.15.131:42468) 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.933+0000 7f1e51600640 1 --2- 172.21.15.131:0/2249836018 >> v2:172.21.15.157:3300/0 conn(0x7f1e4c0d1250 0x7f1e4c15f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.934+0000 7f1e50c00640 1 -- 172.21.15.131:0/2249836018 >> v2:172.21.15.157:3300/0 conn(0x7f1e4c0d1250 msgr2=0x7f1e4c15f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.934+0000 7f1e50c00640 1 --2- 172.21.15.131:0/2249836018 >> v2:172.21.15.157:3300/0 conn(0x7f1e4c0d1250 0x7f1e4c15f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.934+0000 7f1e50c00640 1 -- 172.21.15.131:0/2249836018 >> v1:172.21.15.131:6789/0 conn(0x7f1e4c0b0020 legacy=0x7f1e4c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.934+0000 7f1e50c00640 1 -- 172.21.15.131:0/2249836018 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c163a30 con 0x7f1e4c15cab0 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.934+0000 7f1e4b400640 1 -- 172.21.15.131:0/2249836018 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612423930 0 0) 0x7f1e4c0b0650 con 0x7f1e4c0b0020 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.935+0000 7f1e50c00640 1 --2- 172.21.15.131:0/2249836018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e4c15cab0 0x7f1e4c0afae0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0b3920 tx=0x7f1e4002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a10dadd88f6803 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:49.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.935+0000 7f1e4b400640 1 -- 172.21.15.131:0/2249836018 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40037070 con 0x7f1e4c15cab0 2024-04-28T12:38:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.935+0000 7f1e4b400640 1 -- 172.21.15.131:0/2249836018 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e40032440 con 0x7f1e4c15cab0 2024-04-28T12:38:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.935+0000 7f1e4b400640 1 -- 172.21.15.131:0/2249836018 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40032810 con 0x7f1e4c15cab0 2024-04-28T12:38:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.935+0000 7f1e53600640 1 -- 172.21.15.131:0/2249836018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e4c15cab0 msgr2=0x7f1e4c0afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.936+0000 7f1e53600640 1 --2- 172.21.15.131:0/2249836018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e4c15cab0 0x7f1e4c0afae0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0b3920 tx=0x7f1e4002a1a0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.936+0000 7f1e53600640 1 -- 172.21.15.131:0/2249836018 shutdown_connections 2024-04-28T12:38:49.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.936+0000 7f1e53600640 1 --2- 172.21.15.131:0/2249836018 >> v2:172.21.15.157:3300/0 conn(0x7f1e4c0d1250 0x7f1e4c15f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.936+0000 7f1e53600640 1 --2- 172.21.15.131:0/2249836018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e4c15cab0 0x7f1e4c0afae0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.936+0000 7f1e53600640 1 -- 172.21.15.131:0/2249836018 >> 172.21.15.131:0/2249836018 conn(0x7f1e4c0b2900 msgr2=0x7f1e4c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:49.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.936+0000 7f1e53600640 1 -- 172.21.15.131:0/2249836018 shutdown_connections 2024-04-28T12:38:49.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.936+0000 7f1e53600640 1 -- 172.21.15.131:0/2249836018 wait complete. 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.937+0000 7f1e53600640 1 Processor -- start 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.938+0000 7f1e53600640 1 -- start start 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.938+0000 7f1e53600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e4c0b0020 0x7f1e4c1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.938+0000 7f1e53600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e4c0d1250 0x7f1e4c158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e53600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e4c15cab0 0x7f1e4c06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e53600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c163230 con 0x7f1e4c15cab0 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e53600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c162f50 con 0x7f1e4c0b0020 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e53600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4c15ce90 con 0x7f1e4c0d1250 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e51600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e4c15cab0 0x7f1e4c06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e51600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e4c15cab0 0x7f1e4c06f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42478/0 (socket says 172.21.15.131:42478) 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e51600640 1 -- 172.21.15.131:0/1878448937 learned_addr learned my addr 172.21.15.131:0/1878448937 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e50c00640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e4c0b0020 0x7f1e4c1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e4be00640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e4c0d1250 0x7f1e4c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e51600640 1 -- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e4c0d1250 msgr2=0x7f1e4c158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e51600640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e4c0d1250 0x7f1e4c158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e51600640 1 -- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e4c0b0020 msgr2=0x7f1e4c1567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e51600640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e4c0b0020 0x7f1e4c1567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.939+0000 7f1e51600640 1 -- 172.21.15.131:0/1878448937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c0b2330 con 0x7f1e4c15cab0 2024-04-28T12:38:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.940+0000 7f1e51600640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e4c15cab0 0x7f1e4c06f1c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e44007e40 tx=0x7f1e4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.940+0000 7f1e48c00640 1 -- 172.21.15.131:0/1878448937 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44014620 con 0x7f1e4c15cab0 2024-04-28T12:38:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.940+0000 7f1e48c00640 1 -- 172.21.15.131:0/1878448937 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e44019070 con 0x7f1e4c15cab0 2024-04-28T12:38:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.940+0000 7f1e48c00640 1 -- 172.21.15.131:0/1878448937 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44014a20 con 0x7f1e4c15cab0 2024-04-28T12:38:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.940+0000 7f1e53600640 1 -- 172.21.15.131:0/1878448937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c0b2610 con 0x7f1e4c15cab0 2024-04-28T12:38:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.940+0000 7f1e53600640 1 -- 172.21.15.131:0/1878448937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c15aff0 con 0x7f1e4c15cab0 2024-04-28T12:38:49.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.941+0000 7f1e53600640 1 -- 172.21.15.131:0/1878448937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4c069350 con 0x7f1e4c15cab0 2024-04-28T12:38:49.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.942+0000 7f1e48c00640 1 -- 172.21.15.131:0/1878448937 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e44014bc0 con 0x7f1e4c15cab0 2024-04-28T12:38:49.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.943+0000 7f1e48c00640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1e2c06c920 0x7f1e2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:49.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.943+0000 7f1e48c00640 1 -- 172.21.15.131:0/1878448937 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1e4408c800 con 0x7f1e4c15cab0 2024-04-28T12:38:49.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.944+0000 7f1e50c00640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1e2c06c920 0x7f1e2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:49.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.945+0000 7f1e50c00640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1e2c06c920 0x7f1e2c06ede0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1e40034000 tx=0x7f1e4002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:49.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:49.947+0000 7f1e48c00640 1 -- 172.21.15.131:0/1878448937 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e4405a820 con 0x7f1e4c15cab0 2024-04-28T12:38:50.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.132+0000 7f1e53600640 1 -- 172.21.15.131:0/1878448937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e4c15b860 con 0x7f1e4c15cab0 2024-04-28T12:38:50.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.135+0000 7f1e48c00640 1 -- 172.21.15.131:0/1878448937 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7f1e4405a120 con 0x7f1e4c15cab0 2024-04-28T12:38:50.132 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:50.133 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:50.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.138+0000 7f1e53600640 1 -- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1e2c06c920 msgr2=0x7f1e2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:50.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.138+0000 7f1e53600640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1e2c06c920 0x7f1e2c06ede0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1e40034000 tx=0x7f1e4002e000 comp rx=0 tx=0).stop 2024-04-28T12:38:50.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.138+0000 7f1e53600640 1 -- 172.21.15.131:0/1878448937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e4c15cab0 msgr2=0x7f1e4c06f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:50.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.138+0000 7f1e53600640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e4c15cab0 0x7f1e4c06f1c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e44007e40 tx=0x7f1e4400e4b0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.139+0000 7f1e53600640 1 -- 172.21.15.131:0/1878448937 shutdown_connections 2024-04-28T12:38:50.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.139+0000 7f1e53600640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f1e2c06c920 0x7f1e2c06ede0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.139+0000 7f1e53600640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e4c15cab0 0x7f1e4c06f1c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.139+0000 7f1e53600640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1e4c0d1250 0x7f1e4c158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.139+0000 7f1e53600640 1 --2- 172.21.15.131:0/1878448937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e4c0b0020 0x7f1e4c1567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.139+0000 7f1e53600640 1 -- 172.21.15.131:0/1878448937 >> 172.21.15.131:0/1878448937 conn(0x7f1e4c0b2900 msgr2=0x7f1e4c0b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.139+0000 7f1e53600640 1 -- 172.21.15.131:0/1878448937 shutdown_connections 2024-04-28T12:38:50.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.139+0000 7f1e53600640 1 -- 172.21.15.131:0/1878448937 wait complete. 2024-04-28T12:38:50.155 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:50.155 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:50.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.358+0000 7faa88400640 1 Processor -- start 2024-04-28T12:38:50.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.358+0000 7faa88400640 1 -- start start 2024-04-28T12:38:50.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.360+0000 7faa88400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faa800cf440 0x7faa800b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.360+0000 7faa88400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa8006eaf0 con 0x7faa8015d5f0 2024-04-28T12:38:50.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.360+0000 7faa88400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa8006edd0 con 0x7faa800cf820 2024-04-28T12:38:50.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.360+0000 7faa88400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faa8006f420 con 0x7faa800cf440 2024-04-28T12:38:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.360+0000 7faa86400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7faa8015d5f0 0x7faa8015da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49432/0 (socket says 172.21.15.131:49432) 2024-04-28T12:38:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.360+0000 7faa86400640 1 -- 172.21.15.131:0/609144795 learned_addr learned my addr 172.21.15.131:0/609144795 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.360+0000 7faa85a00640 1 --2- 172.21.15.131:0/609144795 >> v2:172.21.15.157:3300/0 conn(0x7faa800cf440 0x7faa800b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4064678810 0 0) 0x7faa8006eaf0 con 0x7faa8015d5f0 2024-04-28T12:38:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa68003520 con 0x7faa8015d5f0 2024-04-28T12:38:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2934900241 0 0) 0x7faa68003520 con 0x7faa8015d5f0 2024-04-28T12:38:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 >> v2:172.21.15.157:3300/0 conn(0x7faa800cf440 msgr2=0x7faa800b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 --2- 172.21.15.131:0/609144795 >> v2:172.21.15.157:3300/0 conn(0x7faa800cf440 0x7faa800b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 >> v1:172.21.15.157:6789/0 conn(0x7faa800cf820 legacy=0x7faa800cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8006f0b0 con 0x7faa8015d5f0 2024-04-28T12:38:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7faa780019f0 con 0x7faa8015d5f0 2024-04-28T12:38:50.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 >> v1:172.21.15.131:6789/0 conn(0x7faa8015d5f0 legacy=0x7faa8015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 --2- 172.21.15.131:0/609144795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa680039b0 0x7faa68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 --2- 172.21.15.131:0/609144795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa68004800 0x7faa68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 --2- 172.21.15.131:0/609144795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa680051c0 0x7faa6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faa8006f420 con 0x7faa68004800 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faa6800c160 con 0x7faa680039b0 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.361+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faa6800c360 con 0x7faa680051c0 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.362+0000 7faa85000640 1 --2- 172.21.15.131:0/609144795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa68004800 0x7faa68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.362+0000 7faa86400640 1 --2- 172.21.15.131:0/609144795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa680051c0 0x7faa6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.362+0000 7faa85a00640 1 --2- 172.21.15.131:0/609144795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa680039b0 0x7faa68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.362+0000 7faa85000640 1 -- 172.21.15.131:0/609144795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa680051c0 msgr2=0x7faa6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.362+0000 7faa85000640 1 --2- 172.21.15.131:0/609144795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa680051c0 0x7faa6800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.362+0000 7faa85000640 1 -- 172.21.15.131:0/609144795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa680039b0 msgr2=0x7faa68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.362+0000 7faa85000640 1 --2- 172.21.15.131:0/609144795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa680039b0 0x7faa68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.362+0000 7faa85000640 1 -- 172.21.15.131:0/609144795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa68003520 con 0x7faa68004800 2024-04-28T12:38:50.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.362+0000 7faa86400640 1 -- 172.21.15.131:0/609144795 reap_dead start 2024-04-28T12:38:50.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.362+0000 7faa85a00640 1 --2- 172.21.15.131:0/609144795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa680039b0 0x7faa68003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:50.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.363+0000 7faa85000640 1 --2- 172.21.15.131:0/609144795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa68004800 0x7faa68004c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7faa6c001750 tx=0x7faa68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a171816cc00ca1a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:50.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.363+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c001370 con 0x7faa68004800 2024-04-28T12:38:50.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.363+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6c0052d0 con 0x7faa68004800 2024-04-28T12:38:50.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.363+0000 7faa7fe00640 1 -- 172.21.15.131:0/609144795 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c02abc0 con 0x7faa68004800 2024-04-28T12:38:50.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.364+0000 7faa88400640 1 -- 172.21.15.131:0/609144795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa68004800 msgr2=0x7faa68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:50.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.364+0000 7faa88400640 1 --2- 172.21.15.131:0/609144795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa68004800 0x7faa68004c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7faa6c001750 tx=0x7faa68005830 comp rx=0 tx=0).stop 2024-04-28T12:38:50.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.364+0000 7faa88400640 1 -- 172.21.15.131:0/609144795 shutdown_connections 2024-04-28T12:38:50.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.364+0000 7faa88400640 1 --2- 172.21.15.131:0/609144795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa68004800 0x7faa68004c80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.364+0000 7faa88400640 1 -- 172.21.15.131:0/609144795 >> 172.21.15.131:0/609144795 conn(0x7faa800b2900 msgr2=0x7faa800b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:50.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.364+0000 7faa88400640 1 -- 172.21.15.131:0/609144795 shutdown_connections 2024-04-28T12:38:50.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.364+0000 7faa88400640 1 -- 172.21.15.131:0/609144795 wait complete. 2024-04-28T12:38:50.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.366+0000 7faa88400640 1 Processor -- start 2024-04-28T12:38:50.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.366+0000 7faa88400640 1 -- start start 2024-04-28T12:38:50.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.367+0000 7faa88400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa68004800 0x7faa8013cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.367+0000 7faa88400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa8013a100 0x7faa801399d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.367+0000 7faa88400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8015f370 0x7faa8015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.367+0000 7faa88400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faa800b5250 con 0x7faa8013a100 2024-04-28T12:38:50.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.367+0000 7faa88400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faa8006f8b0 con 0x7faa8015f370 2024-04-28T12:38:50.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.367+0000 7faa88400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faa800b53d0 con 0x7faa68004800 2024-04-28T12:38:50.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.367+0000 7faa85000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa8013a100 0x7faa801399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.367+0000 7faa85000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa8013a100 0x7faa801399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42508/0 (socket says 172.21.15.131:42508) 2024-04-28T12:38:50.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.367+0000 7faa85000640 1 -- 172.21.15.131:0/3600095884 learned_addr learned my addr 172.21.15.131:0/3600095884 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:50.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.368+0000 7faa86400640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8015f370 0x7faa8015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.368+0000 7faa85a00640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa68004800 0x7faa8013cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.368+0000 7faa85000640 1 -- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa68004800 msgr2=0x7faa8013cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.368+0000 7faa85000640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa68004800 0x7faa8013cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.368+0000 7faa85000640 1 -- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8015f370 msgr2=0x7faa8015f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.368+0000 7faa85000640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8015f370 0x7faa8015f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.368+0000 7faa85000640 1 -- 172.21.15.131:0/3600095884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa80139f10 con 0x7faa8013a100 2024-04-28T12:38:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.368+0000 7faa85000640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa8013a100 0x7faa801399d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7faa6c001870 tx=0x7faa6c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.368+0000 7faa7d600640 1 -- 172.21.15.131:0/3600095884 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c03f020 con 0x7faa8013a100 2024-04-28T12:38:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.368+0000 7faa7d600640 1 -- 172.21.15.131:0/3600095884 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6c005480 con 0x7faa8013a100 2024-04-28T12:38:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.368+0000 7faa7d600640 1 -- 172.21.15.131:0/3600095884 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c037040 con 0x7faa8013a100 2024-04-28T12:38:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.368+0000 7faa88400640 1 -- 172.21.15.131:0/3600095884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8013d210 con 0x7faa8013a100 2024-04-28T12:38:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.369+0000 7faa88400640 1 -- 172.21.15.131:0/3600095884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa8013f090 con 0x7faa8013a100 2024-04-28T12:38:50.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.370+0000 7faa88400640 1 -- 172.21.15.131:0/3600095884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa80069350 con 0x7faa8013a100 2024-04-28T12:38:50.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.371+0000 7faa7d600640 1 -- 172.21.15.131:0/3600095884 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa6c042020 con 0x7faa8013a100 2024-04-28T12:38:50.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.372+0000 7faa7d600640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faa6006c8f0 0x7faa6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.372+0000 7faa7d600640 1 -- 172.21.15.131:0/3600095884 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7faa6c0abb00 con 0x7faa8013a100 2024-04-28T12:38:50.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.372+0000 7faa85a00640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faa6006c8f0 0x7faa6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.373+0000 7faa85a00640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faa6006c8f0 0x7faa6006edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faa80069a00 tx=0x7faa74008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.376+0000 7faa7d600640 1 -- 172.21.15.131:0/3600095884 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa6c049030 con 0x7faa8013a100 2024-04-28T12:38:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.575+0000 7faa88400640 1 -- 172.21.15.131:0/3600095884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa80146120 con 0x7faa8013a100 2024-04-28T12:38:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.576+0000 7faa7d600640 1 -- 172.21.15.131:0/3600095884 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7faa6c079b30 con 0x7faa8013a100 2024-04-28T12:38:50.573 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:50.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.578+0000 7faa88400640 1 -- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faa6006c8f0 msgr2=0x7faa6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.578+0000 7faa88400640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faa6006c8f0 0x7faa6006edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faa80069a00 tx=0x7faa74008040 comp rx=0 tx=0).stop 2024-04-28T12:38:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.578+0000 7faa88400640 1 -- 172.21.15.131:0/3600095884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa8013a100 msgr2=0x7faa801399d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.578+0000 7faa88400640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa8013a100 0x7faa801399d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7faa6c001870 tx=0x7faa6c001120 comp rx=0 tx=0).stop 2024-04-28T12:38:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.579+0000 7faa88400640 1 -- 172.21.15.131:0/3600095884 shutdown_connections 2024-04-28T12:38:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.579+0000 7faa88400640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faa6006c8f0 0x7faa6006edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.579+0000 7faa88400640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8015f370 0x7faa8015f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.579+0000 7faa88400640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa8013a100 0x7faa801399d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.579+0000 7faa88400640 1 --2- 172.21.15.131:0/3600095884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa68004800 0x7faa8013cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.579+0000 7faa88400640 1 -- 172.21.15.131:0/3600095884 >> 172.21.15.131:0/3600095884 conn(0x7faa800b2900 msgr2=0x7faa800b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.579+0000 7faa88400640 1 -- 172.21.15.131:0/3600095884 shutdown_connections 2024-04-28T12:38:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.579+0000 7faa88400640 1 -- 172.21.15.131:0/3600095884 wait complete. 2024-04-28T12:38:50.593 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:50.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.800+0000 7fb3e3000640 1 Processor -- start 2024-04-28T12:38:50.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.800+0000 7fb3e3000640 1 -- start start 2024-04-28T12:38:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.802+0000 7fb3e3000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb3dc15a7e0 0x7fb3dc0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.802+0000 7fb3e3000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb3dc15d9c0 0x7fb3dc0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.802+0000 7fb3e3000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3dc0b3900 con 0x7fb3dc0b5410 2024-04-28T12:38:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.802+0000 7fb3e3000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb3dc15abc0 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.802+0000 7fb3e3000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb3dc0b3b70 con 0x7fb3dc15a7e0 2024-04-28T12:38:50.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.802+0000 7fb3e2600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb3dc15d9c0 0x7fb3dc0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.802+0000 7fb3e2600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb3dc15d9c0 0x7fb3dc0b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41374/0 (socket says 172.21.15.131:41374) 2024-04-28T12:38:50.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.802+0000 7fb3e2600640 1 -- 172.21.15.131:0/1185577066 learned_addr learned my addr 172.21.15.131:0/1185577066 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:50.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.802+0000 7fb3e1c00640 1 --2- 172.21.15.131:0/1185577066 >> v2:172.21.15.157:3300/0 conn(0x7fb3dc15a7e0 0x7fb3dc0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.803+0000 7fb3e2600640 1 -- 172.21.15.131:0/1185577066 >> v2:172.21.15.157:3300/0 conn(0x7fb3dc15a7e0 msgr2=0x7fb3dc0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:50.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.803+0000 7fb3e2600640 1 --2- 172.21.15.131:0/1185577066 >> v2:172.21.15.157:3300/0 conn(0x7fb3dc15a7e0 0x7fb3dc0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.803+0000 7fb3e2600640 1 -- 172.21.15.131:0/1185577066 >> v1:172.21.15.157:6789/0 conn(0x7fb3dc0b5410 legacy=0x7fb3dc15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:50.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.803+0000 7fb3e2600640 1 -- 172.21.15.131:0/1185577066 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3dc0b3e50 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.804+0000 7fb3e2600640 1 --2- 172.21.15.131:0/1185577066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3dc15d9c0 0x7fb3dc0b5890 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0d1620 tx=0x7fb3d402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec791a2353657d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:50.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.804+0000 7fb3dbe00640 1 -- 172.21.15.131:0/1185577066 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4036070 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.804+0000 7fb3dbe00640 1 -- 172.21.15.131:0/1185577066 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d4031600 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.804+0000 7fb3dbe00640 1 -- 172.21.15.131:0/1185577066 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40319d0 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.805+0000 7fb3e3000640 1 -- 172.21.15.131:0/1185577066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3dc15d9c0 msgr2=0x7fb3dc0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:50.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.805+0000 7fb3e3000640 1 --2- 172.21.15.131:0/1185577066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3dc15d9c0 0x7fb3dc0b5890 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0d1620 tx=0x7fb3d402a6f0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.805+0000 7fb3e3000640 1 -- 172.21.15.131:0/1185577066 shutdown_connections 2024-04-28T12:38:50.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.805+0000 7fb3e3000640 1 --2- 172.21.15.131:0/1185577066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3dc15d9c0 0x7fb3dc0b5890 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.805+0000 7fb3e3000640 1 --2- 172.21.15.131:0/1185577066 >> v2:172.21.15.157:3300/0 conn(0x7fb3dc15a7e0 0x7fb3dc0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.805+0000 7fb3e3000640 1 -- 172.21.15.131:0/1185577066 >> 172.21.15.131:0/1185577066 conn(0x7fb3dc068090 msgr2=0x7fb3dc0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:50.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.805+0000 7fb3e3000640 1 -- 172.21.15.131:0/1185577066 shutdown_connections 2024-04-28T12:38:50.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.805+0000 7fb3e3000640 1 -- 172.21.15.131:0/1185577066 wait complete. 2024-04-28T12:38:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.807+0000 7fb3e3000640 1 Processor -- start 2024-04-28T12:38:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.807+0000 7fb3e3000640 1 -- start start 2024-04-28T12:38:50.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e3000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3dc0b5410 0x7fb3dc0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e3000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3dc15a7e0 0x7fb3dc14ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e3000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3dc15d9c0 0x7fb3dc14f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e3000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc0b4b90 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e3000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc15e010 con 0x7fb3dc15a7e0 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e3000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb3dc15e2e0 con 0x7fb3dc0b5410 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e2600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3dc15d9c0 0x7fb3dc14f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e2600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3dc15d9c0 0x7fb3dc14f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41378/0 (socket says 172.21.15.131:41378) 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e2600640 1 -- 172.21.15.131:0/3024387650 learned_addr learned my addr 172.21.15.131:0/3024387650 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e1c00640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3dc0b5410 0x7fb3dc0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e1200640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3dc15a7e0 0x7fb3dc14ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e2600640 1 -- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3dc0b5410 msgr2=0x7fb3dc0c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e2600640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3dc0b5410 0x7fb3dc0c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e2600640 1 -- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3dc15a7e0 msgr2=0x7fb3dc14ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e2600640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3dc15a7e0 0x7fb3dc14ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.808+0000 7fb3e2600640 1 -- 172.21.15.131:0/3024387650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3dc0cf810 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.809+0000 7fb3e2600640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3dc15d9c0 0x7fb3dc14f260 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009f40 tx=0x7fb3d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:50.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.809+0000 7fb3d9600640 1 -- 172.21.15.131:0/3024387650 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4036050 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.809+0000 7fb3d9600640 1 -- 172.21.15.131:0/3024387650 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d4042070 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.809+0000 7fb3e3000640 1 -- 172.21.15.131:0/3024387650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3dc0cfaf0 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.809+0000 7fb3d9600640 1 -- 172.21.15.131:0/3024387650 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4009510 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.809+0000 7fb3e3000640 1 -- 172.21.15.131:0/3024387650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3dc0cf480 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.810+0000 7fb3e3000640 1 -- 172.21.15.131:0/3024387650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3dc0506c0 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.811+0000 7fb3d9600640 1 -- 172.21.15.131:0/3024387650 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3d4007bd0 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.812+0000 7fb3d9600640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:50.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.812+0000 7fb3d9600640 1 -- 172.21.15.131:0/3024387650 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb3d40ace80 con 0x7fb3dc15d9c0 2024-04-28T12:38:50.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.813+0000 7fb3e1c00640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:50.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.814+0000 7fb3e1c00640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0006fd0 tx=0x7fb3d0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:50.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:50.817+0000 7fb3d9600640 1 -- 172.21.15.131:0/3024387650 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb3d4045360 con 0x7fb3dc15d9c0 2024-04-28T12:38:51.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.021+0000 7fb3e3000640 1 -- 172.21.15.131:0/3024387650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb3dc0c5d10 con 0x7fb3dc15d9c0 2024-04-28T12:38:51.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.022+0000 7fb3d9600640 1 -- 172.21.15.131:0/3024387650 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fb3d407aea0 con 0x7fb3dc15d9c0 2024-04-28T12:38:51.019 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:51.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.025+0000 7fb3e3000640 1 -- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb3bc06c8f0 msgr2=0x7fb3bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.025+0000 7fb3e3000640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0006fd0 tx=0x7fb3d0008040 comp rx=0 tx=0).stop 2024-04-28T12:38:51.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.025+0000 7fb3e3000640 1 -- 172.21.15.131:0/3024387650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3dc15d9c0 msgr2=0x7fb3dc14f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.025+0000 7fb3e3000640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3dc15d9c0 0x7fb3dc14f260 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009f40 tx=0x7fb3d4004a40 comp rx=0 tx=0).stop 2024-04-28T12:38:51.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.025+0000 7fb3e3000640 1 -- 172.21.15.131:0/3024387650 shutdown_connections 2024-04-28T12:38:51.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.025+0000 7fb3e3000640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb3bc06c8f0 0x7fb3bc06edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.025+0000 7fb3e3000640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3dc15d9c0 0x7fb3dc14f260 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.025+0000 7fb3e3000640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3dc15a7e0 0x7fb3dc14ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.025+0000 7fb3e3000640 1 --2- 172.21.15.131:0/3024387650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3dc0b5410 0x7fb3dc0c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.025+0000 7fb3e3000640 1 -- 172.21.15.131:0/3024387650 >> 172.21.15.131:0/3024387650 conn(0x7fb3dc068090 msgr2=0x7fb3dc068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.025+0000 7fb3e3000640 1 -- 172.21.15.131:0/3024387650 shutdown_connections 2024-04-28T12:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.026+0000 7fb3e3000640 1 -- 172.21.15.131:0/3024387650 wait complete. 2024-04-28T12:38:51.040 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:51.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.247+0000 7fda24a00640 1 Processor -- start 2024-04-28T12:38:51.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.247+0000 7fda24a00640 1 -- start start 2024-04-28T12:38:51.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.249+0000 7fda24a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda201572b0 0x7fda200b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.249+0000 7fda24a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fda200d0bd0 0x7fda200d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.249+0000 7fda24a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda2015d2e0 con 0x7fda200af8a0 2024-04-28T12:38:51.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.249+0000 7fda24a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fda2015d5c0 con 0x7fda200d0bd0 2024-04-28T12:38:51.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.249+0000 7fda24a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fda2015d8a0 con 0x7fda201572b0 2024-04-28T12:38:51.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.249+0000 7fda17e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fda200d0bd0 0x7fda200d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.249+0000 7fda1fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fda200af8a0 0x7fda200afd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35080/0 (socket says 172.21.15.131:35080) 2024-04-28T12:38:51.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.249+0000 7fda1fe00640 1 -- 172.21.15.131:0/2442222942 learned_addr learned my addr 172.21.15.131:0/2442222942 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:51.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.249+0000 7fda17e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fda200d0bd0 0x7fda200d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41392/0 (socket says 172.21.15.131:41392) 2024-04-28T12:38:51.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.249+0000 7fda1f400640 1 --2- 172.21.15.131:0/2442222942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda201572b0 0x7fda200b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.250+0000 7fda17e00640 1 -- 172.21.15.131:0/2442222942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda201572b0 msgr2=0x7fda200b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:51.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.250+0000 7fda17e00640 1 --2- 172.21.15.131:0/2442222942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda201572b0 0x7fda200b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.250+0000 7fda17e00640 1 -- 172.21.15.131:0/2442222942 >> v1:172.21.15.131:6789/0 conn(0x7fda200af8a0 legacy=0x7fda200afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.250+0000 7fda1ea00640 1 -- 172.21.15.131:0/2442222942 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1961057043 0 0) 0x7fda2015d2e0 con 0x7fda200af8a0 2024-04-28T12:38:51.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.250+0000 7fda17e00640 1 -- 172.21.15.131:0/2442222942 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2015db80 con 0x7fda200d0bd0 2024-04-28T12:38:51.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.251+0000 7fda17e00640 1 --2- 172.21.15.131:0/2442222942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda200d0bd0 0x7fda200d1050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fda080097a0 tx=0x7fda0802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6834875e80762ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:51.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.251+0000 7fda1ea00640 1 -- 172.21.15.131:0/2442222942 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08037070 con 0x7fda200d0bd0 2024-04-28T12:38:51.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.251+0000 7fda1ea00640 1 -- 172.21.15.131:0/2442222942 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0802adb0 con 0x7fda200d0bd0 2024-04-28T12:38:51.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.251+0000 7fda1ea00640 1 -- 172.21.15.131:0/2442222942 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08032660 con 0x7fda200d0bd0 2024-04-28T12:38:51.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.252+0000 7fda24a00640 1 -- 172.21.15.131:0/2442222942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda200d0bd0 msgr2=0x7fda200d1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.252+0000 7fda24a00640 1 --2- 172.21.15.131:0/2442222942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda200d0bd0 0x7fda200d1050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fda080097a0 tx=0x7fda0802a060 comp rx=0 tx=0).stop 2024-04-28T12:38:51.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.252+0000 7fda24a00640 1 -- 172.21.15.131:0/2442222942 shutdown_connections 2024-04-28T12:38:51.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.252+0000 7fda24a00640 1 --2- 172.21.15.131:0/2442222942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda200d0bd0 0x7fda200d1050 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.252+0000 7fda24a00640 1 --2- 172.21.15.131:0/2442222942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda201572b0 0x7fda200b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.252+0000 7fda24a00640 1 -- 172.21.15.131:0/2442222942 >> 172.21.15.131:0/2442222942 conn(0x7fda200684e0 msgr2=0x7fda200688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:51.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.252+0000 7fda24a00640 1 -- 172.21.15.131:0/2442222942 shutdown_connections 2024-04-28T12:38:51.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.252+0000 7fda24a00640 1 -- 172.21.15.131:0/2442222942 wait complete. 2024-04-28T12:38:51.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.254+0000 7fda24a00640 1 Processor -- start 2024-04-28T12:38:51.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.254+0000 7fda24a00640 1 -- start start 2024-04-28T12:38:51.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.255+0000 7fda24a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda200af8a0 0x7fda20146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.255+0000 7fda24a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda200d0bd0 0x7fda20150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.255+0000 7fda24a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda201572b0 0x7fda200ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.255+0000 7fda24a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda20160c00 con 0x7fda200af8a0 2024-04-28T12:38:51.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.255+0000 7fda24a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda20160a80 con 0x7fda200d0bd0 2024-04-28T12:38:51.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.255+0000 7fda24a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fda20160d80 con 0x7fda201572b0 2024-04-28T12:38:51.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.255+0000 7fda1f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda200af8a0 0x7fda20146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.255+0000 7fda1f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda200af8a0 0x7fda20146330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41406/0 (socket says 172.21.15.131:41406) 2024-04-28T12:38:51.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.255+0000 7fda1f400640 1 -- 172.21.15.131:0/3640405651 learned_addr learned my addr 172.21.15.131:0/3640405651 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:51.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.255+0000 7fda1fe00640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda201572b0 0x7fda200ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.255+0000 7fda17e00640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda200d0bd0 0x7fda20150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.255+0000 7fda1f400640 1 -- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda201572b0 msgr2=0x7fda200ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.256+0000 7fda1f400640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda201572b0 0x7fda200ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.256+0000 7fda1f400640 1 -- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda200d0bd0 msgr2=0x7fda20150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.256+0000 7fda1f400640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda200d0bd0 0x7fda20150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.256+0000 7fda1f400640 1 -- 172.21.15.131:0/3640405651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda20151500 con 0x7fda200af8a0 2024-04-28T12:38:51.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.256+0000 7fda1f400640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda200af8a0 0x7fda20146330 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fda1000d5f0 tx=0x7fda1000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:51.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.256+0000 7fda17400640 1 -- 172.21.15.131:0/3640405651 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda100125f0 con 0x7fda200af8a0 2024-04-28T12:38:51.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.256+0000 7fda17400640 1 -- 172.21.15.131:0/3640405651 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda10019070 con 0x7fda200af8a0 2024-04-28T12:38:51.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.256+0000 7fda17400640 1 -- 172.21.15.131:0/3640405651 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda100129d0 con 0x7fda200af8a0 2024-04-28T12:38:51.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.256+0000 7fda24a00640 1 -- 172.21.15.131:0/3640405651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda200b0a60 con 0x7fda200af8a0 2024-04-28T12:38:51.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.256+0000 7fda24a00640 1 -- 172.21.15.131:0/3640405651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda200b1020 con 0x7fda200af8a0 2024-04-28T12:38:51.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.257+0000 7fda24a00640 1 -- 172.21.15.131:0/3640405651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda200cd410 con 0x7fda200af8a0 2024-04-28T12:38:51.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.258+0000 7fda17400640 1 -- 172.21.15.131:0/3640405651 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda10012b70 con 0x7fda200af8a0 2024-04-28T12:38:51.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.259+0000 7fda17400640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.259+0000 7fda17400640 1 -- 172.21.15.131:0/3640405651 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fda1008c320 con 0x7fda200af8a0 2024-04-28T12:38:51.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.260+0000 7fda17e00640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.261+0000 7fda17e00640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda08034000 tx=0x7fda0802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:51.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.264+0000 7fda17400640 1 -- 172.21.15.131:0/3640405651 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda200cd410 con 0x7fda200af8a0 2024-04-28T12:38:51.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.466+0000 7fda24a00640 1 -- 172.21.15.131:0/3640405651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fda200506c0 con 0x7fda200af8a0 2024-04-28T12:38:51.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.467+0000 7fda17400640 1 -- 172.21.15.131:0/3640405651 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fda1005a340 con 0x7fda200af8a0 2024-04-28T12:38:51.464 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:51.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.470+0000 7fda24a00640 1 -- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd9fc06c8f0 msgr2=0x7fd9fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.470+0000 7fda24a00640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda08034000 tx=0x7fda0802e000 comp rx=0 tx=0).stop 2024-04-28T12:38:51.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.470+0000 7fda24a00640 1 -- 172.21.15.131:0/3640405651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda200af8a0 msgr2=0x7fda20146330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.470+0000 7fda24a00640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda200af8a0 0x7fda20146330 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fda1000d5f0 tx=0x7fda1000dab0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.470+0000 7fda24a00640 1 -- 172.21.15.131:0/3640405651 shutdown_connections 2024-04-28T12:38:51.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.470+0000 7fda24a00640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd9fc06c8f0 0x7fd9fc06edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.470+0000 7fda24a00640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda201572b0 0x7fda200ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.470+0000 7fda24a00640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda200d0bd0 0x7fda20150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.470+0000 7fda24a00640 1 --2- 172.21.15.131:0/3640405651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda200af8a0 0x7fda20146330 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.470+0000 7fda24a00640 1 -- 172.21.15.131:0/3640405651 >> 172.21.15.131:0/3640405651 conn(0x7fda200684e0 msgr2=0x7fda201554d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:51.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.471+0000 7fda24a00640 1 -- 172.21.15.131:0/3640405651 shutdown_connections 2024-04-28T12:38:51.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.471+0000 7fda24a00640 1 -- 172.21.15.131:0/3640405651 wait complete. 2024-04-28T12:38:51.487 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:51.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.693+0000 7fe025a00640 1 Processor -- start 2024-04-28T12:38:51.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.693+0000 7fe025a00640 1 -- start start 2024-04-28T12:38:51.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.695+0000 7fe025a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0200cf440 0x7fe0200b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.695+0000 7fe025a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe02006eaf0 con 0x7fe02015d5f0 2024-04-28T12:38:51.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.695+0000 7fe025a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe02006edd0 con 0x7fe0200cf820 2024-04-28T12:38:51.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.695+0000 7fe025a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe02006f420 con 0x7fe0200cf440 2024-04-28T12:38:51.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.695+0000 7fe01f400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe02015d5f0 0x7fe02015da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35084/0 (socket says 172.21.15.131:35084) 2024-04-28T12:38:51.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.695+0000 7fe01f400640 1 -- 172.21.15.131:0/339453681 learned_addr learned my addr 172.21.15.131:0/339453681 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:51.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.695+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4253859633 0 0) 0x7fe02006eaf0 con 0x7fe02015d5f0 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.695+0000 7fe01ea00640 1 --2- 172.21.15.131:0/339453681 >> v2:172.21.15.157:3300/0 conn(0x7fe0200cf440 0x7fe0200b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe004003520 con 0x7fe02015d5f0 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4077358679 0 0) 0x7fe004003520 con 0x7fe02015d5f0 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 >> v2:172.21.15.157:3300/0 conn(0x7fe0200cf440 msgr2=0x7fe0200b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 --2- 172.21.15.131:0/339453681 >> v2:172.21.15.157:3300/0 conn(0x7fe0200cf440 0x7fe0200b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 >> v1:172.21.15.157:6789/0 conn(0x7fe0200cf820 legacy=0x7fe0200cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02006f0b0 con 0x7fe02015d5f0 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fe018001670 con 0x7fe02015d5f0 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 >> v1:172.21.15.131:6789/0 conn(0x7fe02015d5f0 legacy=0x7fe02015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 --2- 172.21.15.131:0/339453681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0040039b0 0x7fe004003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 --2- 172.21.15.131:0/339453681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004004800 0x7fe004004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 --2- 172.21.15.131:0/339453681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0040052e0 0x7fe00400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe02006f420 con 0x7fe004004800 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe00400c190 con 0x7fe0040052e0 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe00400c390 con 0x7fe0040039b0 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.696+0000 7fe016000640 1 --2- 172.21.15.131:0/339453681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004004800 0x7fe004004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.697+0000 7fe01ea00640 1 --2- 172.21.15.131:0/339453681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0040039b0 0x7fe004003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.697+0000 7fe016000640 1 -- 172.21.15.131:0/339453681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0040039b0 msgr2=0x7fe004003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.697+0000 7fe016000640 1 --2- 172.21.15.131:0/339453681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0040039b0 0x7fe004003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.697+0000 7fe016000640 1 -- 172.21.15.131:0/339453681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0040052e0 msgr2=0x7fe00400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.697+0000 7fe016000640 1 --2- 172.21.15.131:0/339453681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0040052e0 0x7fe00400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.697+0000 7fe016000640 1 -- 172.21.15.131:0/339453681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe004003520 con 0x7fe004004800 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.697+0000 7fe01f400640 1 -- 172.21.15.131:0/339453681 reap_dead start 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.698+0000 7fe016000640 1 --2- 172.21.15.131:0/339453681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004004800 0x7fe004004c80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe008003d80 tx=0x7fe004005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc97c6f62eedb23 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.698+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008037070 con 0x7fe004004800 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.698+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe00802a9d0 con 0x7fe004004800 2024-04-28T12:38:51.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.698+0000 7fe01e000640 1 -- 172.21.15.131:0/339453681 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008003980 con 0x7fe004004800 2024-04-28T12:38:51.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.699+0000 7fe025a00640 1 -- 172.21.15.131:0/339453681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004004800 msgr2=0x7fe004004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.699+0000 7fe025a00640 1 --2- 172.21.15.131:0/339453681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004004800 0x7fe004004c80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe008003d80 tx=0x7fe004005830 comp rx=0 tx=0).stop 2024-04-28T12:38:51.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.699+0000 7fe025a00640 1 -- 172.21.15.131:0/339453681 shutdown_connections 2024-04-28T12:38:51.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.699+0000 7fe025a00640 1 --2- 172.21.15.131:0/339453681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe004004800 0x7fe004004c80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.699+0000 7fe025a00640 1 -- 172.21.15.131:0/339453681 >> 172.21.15.131:0/339453681 conn(0x7fe0200b2900 msgr2=0x7fe0200b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:51.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.699+0000 7fe025a00640 1 -- 172.21.15.131:0/339453681 shutdown_connections 2024-04-28T12:38:51.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.699+0000 7fe025a00640 1 -- 172.21.15.131:0/339453681 wait complete. 2024-04-28T12:38:51.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.701+0000 7fe025a00640 1 Processor -- start 2024-04-28T12:38:51.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.701+0000 7fe025a00640 1 -- start start 2024-04-28T12:38:51.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.702+0000 7fe025a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe004004800 0x7fe02013b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.702+0000 7fe025a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe020145970 0x7fe02013fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.702+0000 7fe025a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe020145df0 0x7fe02015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.702+0000 7fe025a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe0200b5250 con 0x7fe020145df0 2024-04-28T12:38:51.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.702+0000 7fe025a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe02006f8b0 con 0x7fe020145970 2024-04-28T12:38:51.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.702+0000 7fe025a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe0200b53d0 con 0x7fe004004800 2024-04-28T12:38:51.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.702+0000 7fe01f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe020145df0 0x7fe02015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.702+0000 7fe01f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe020145df0 0x7fe02015bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41412/0 (socket says 172.21.15.131:41412) 2024-04-28T12:38:51.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.702+0000 7fe01f400640 1 -- 172.21.15.131:0/852594900 learned_addr learned my addr 172.21.15.131:0/852594900 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.702+0000 7fe01ea00640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe004004800 0x7fe02013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.702+0000 7fe016000640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe020145970 0x7fe02013fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.703+0000 7fe01f400640 1 -- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe004004800 msgr2=0x7fe02013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.703+0000 7fe01f400640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe004004800 0x7fe02013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.703+0000 7fe01f400640 1 -- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe020145970 msgr2=0x7fe02013fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.703+0000 7fe01f400640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe020145970 0x7fe02013fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.703+0000 7fe01f400640 1 -- 172.21.15.131:0/852594900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe020140670 con 0x7fe020145df0 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.703+0000 7fe016000640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe020145970 0x7fe02013fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.703+0000 7fe01f400640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe020145df0 0x7fe02015bc70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe02015d9d0 tx=0x7fe01800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.703+0000 7fe017400640 1 -- 172.21.15.131:0/852594900 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe018007230 con 0x7fe020145df0 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.703+0000 7fe017400640 1 -- 172.21.15.131:0/852594900 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0180073f0 con 0x7fe020145df0 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.703+0000 7fe017400640 1 -- 172.21.15.131:0/852594900 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0180047e0 con 0x7fe020145df0 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.703+0000 7fe025a00640 1 -- 172.21.15.131:0/852594900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe02013ab50 con 0x7fe020145df0 2024-04-28T12:38:51.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.703+0000 7fe025a00640 1 -- 172.21.15.131:0/852594900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe02013b110 con 0x7fe020145df0 2024-04-28T12:38:51.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.704+0000 7fe025a00640 1 -- 172.21.15.131:0/852594900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0200cef20 con 0x7fe020145df0 2024-04-28T12:38:51.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.705+0000 7fe017400640 1 -- 172.21.15.131:0/852594900 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe018011070 con 0x7fe020145df0 2024-04-28T12:38:51.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.706+0000 7fe017400640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdffc06c8f0 0x7fdffc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:51.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.706+0000 7fe017400640 1 -- 172.21.15.131:0/852594900 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe01808b690 con 0x7fe020145df0 2024-04-28T12:38:51.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.707+0000 7fe01ea00640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdffc06c8f0 0x7fdffc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:51.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.708+0000 7fe01ea00640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdffc06c8f0 0x7fdffc06edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe020069a00 tx=0x7fe01000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:51.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.711+0000 7fe017400640 1 -- 172.21.15.131:0/852594900 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe0200cef20 con 0x7fe020145df0 2024-04-28T12:38:51.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.910+0000 7fe025a00640 1 -- 172.21.15.131:0/852594900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe02013b2e0 con 0x7fe020145df0 2024-04-28T12:38:51.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.911+0000 7fe017400640 1 -- 172.21.15.131:0/852594900 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fe0180596e0 con 0x7fe020145df0 2024-04-28T12:38:51.908 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.914+0000 7fe025a00640 1 -- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdffc06c8f0 msgr2=0x7fdffc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.914+0000 7fe025a00640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdffc06c8f0 0x7fdffc06edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe020069a00 tx=0x7fe01000b040 comp rx=0 tx=0).stop 2024-04-28T12:38:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.914+0000 7fe025a00640 1 -- 172.21.15.131:0/852594900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe020145df0 msgr2=0x7fe02015bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.914+0000 7fe025a00640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe020145df0 0x7fe02015bc70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe02015d9d0 tx=0x7fe01800cb90 comp rx=0 tx=0).stop 2024-04-28T12:38:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.914+0000 7fe025a00640 1 -- 172.21.15.131:0/852594900 shutdown_connections 2024-04-28T12:38:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.914+0000 7fe025a00640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdffc06c8f0 0x7fdffc06edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.914+0000 7fe025a00640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe020145df0 0x7fe02015bc70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.914+0000 7fe025a00640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe020145970 0x7fe02013fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.914+0000 7fe025a00640 1 --2- 172.21.15.131:0/852594900 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe004004800 0x7fe02013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.914+0000 7fe025a00640 1 -- 172.21.15.131:0/852594900 >> 172.21.15.131:0/852594900 conn(0x7fe0200b2900 msgr2=0x7fe0200b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.914+0000 7fe025a00640 1 -- 172.21.15.131:0/852594900 shutdown_connections 2024-04-28T12:38:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:51.914+0000 7fe025a00640 1 -- 172.21.15.131:0/852594900 wait complete. 2024-04-28T12:38:51.929 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T12:38:52.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.135+0000 7f9dd0c00640 1 Processor -- start 2024-04-28T12:38:52.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.135+0000 7f9dd0c00640 1 -- start start 2024-04-28T12:38:52.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.136+0000 7f9dd0c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9dcc145590 0x7f9dcc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:52.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.136+0000 7f9dd0c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dcc0cf060 con 0x7f9dcc0b3490 2024-04-28T12:38:52.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.136+0000 7f9dd0c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dcc0cf340 con 0x7f9dcc06f730 2024-04-28T12:38:52.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.136+0000 7f9dd0c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9dcc0d0100 con 0x7f9dcc145590 2024-04-28T12:38:52.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.137+0000 7f9dc9e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9dcc145590 0x7f9dcc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:52.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.137+0000 7f9dc3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9dcc0b3490 0x7f9dcc06f020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35088/0 (socket says 172.21.15.131:35088) 2024-04-28T12:38:52.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.137+0000 7f9dc3e00640 1 -- 172.21.15.131:0/882953403 learned_addr learned my addr 172.21.15.131:0/882953403 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:52.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.137+0000 7f9dc9e00640 1 -- 172.21.15.131:0/882953403 >> v1:172.21.15.157:6789/0 conn(0x7f9dcc06f730 legacy=0x7f9dcc071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:52.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.137+0000 7f9dc9e00640 1 -- 172.21.15.131:0/882953403 >> v1:172.21.15.131:6789/0 conn(0x7f9dcc0b3490 legacy=0x7f9dcc06f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:52.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.137+0000 7f9dc9e00640 1 -- 172.21.15.131:0/882953403 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc0cf620 con 0x7f9dcc145590 2024-04-28T12:38:52.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.137+0000 7f9dc9400640 1 -- 172.21.15.131:0/882953403 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1145406254 0 0) 0x7f9dcc0cf060 con 0x7f9dcc0b3490 2024-04-28T12:38:52.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.138+0000 7f9dc9e00640 1 --2- 172.21.15.131:0/882953403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dcc145590 0x7f9dcc0ce780 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0097a0 tx=0x7f9dbc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5d92820b06a65d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:52.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.138+0000 7f9dc9400640 1 -- 172.21.15.131:0/882953403 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc037070 con 0x7f9dcc145590 2024-04-28T12:38:52.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.139+0000 7f9dc9400640 1 -- 172.21.15.131:0/882953403 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dbc032440 con 0x7f9dcc145590 2024-04-28T12:38:52.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.139+0000 7f9dc9400640 1 -- 172.21.15.131:0/882953403 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc032810 con 0x7f9dcc145590 2024-04-28T12:38:52.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.139+0000 7f9dd0c00640 1 -- 172.21.15.131:0/882953403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dcc145590 msgr2=0x7f9dcc0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:52.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.139+0000 7f9dd0c00640 1 --2- 172.21.15.131:0/882953403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dcc145590 0x7f9dcc0ce780 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0097a0 tx=0x7f9dbc02a1e0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.139+0000 7f9dd0c00640 1 -- 172.21.15.131:0/882953403 shutdown_connections 2024-04-28T12:38:52.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.139+0000 7f9dd0c00640 1 --2- 172.21.15.131:0/882953403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dcc145590 0x7f9dcc0ce780 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.139+0000 7f9dd0c00640 1 -- 172.21.15.131:0/882953403 >> 172.21.15.131:0/882953403 conn(0x7f9dcc0b2900 msgr2=0x7f9dcc0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:52.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.139+0000 7f9dd0c00640 1 -- 172.21.15.131:0/882953403 shutdown_connections 2024-04-28T12:38:52.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.139+0000 7f9dd0c00640 1 -- 172.21.15.131:0/882953403 wait complete. 2024-04-28T12:38:52.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.141+0000 7f9dd0c00640 1 Processor -- start 2024-04-28T12:38:52.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.141+0000 7f9dd0c00640 1 -- start start 2024-04-28T12:38:52.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.142+0000 7f9dd0c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc06f730 0x7f9dcc1e4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:52.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.142+0000 7f9dd0c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dcc0b3490 0x7f9dcc140020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:52.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.142+0000 7f9dd0c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dcc145590 0x7f9dcc13f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:52.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.142+0000 7f9dd0c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc145970 con 0x7f9dcc0b3490 2024-04-28T12:38:52.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.142+0000 7f9dd0c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc0cfbe0 con 0x7f9dcc06f730 2024-04-28T12:38:52.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.142+0000 7f9dd0c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9dcc0cecc0 con 0x7f9dcc145590 2024-04-28T12:38:52.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.142+0000 7f9dc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dcc0b3490 0x7f9dcc140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:52.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.143+0000 7f9dc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dcc0b3490 0x7f9dcc140020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41432/0 (socket says 172.21.15.131:41432) 2024-04-28T12:38:52.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.143+0000 7f9dc3e00640 1 -- 172.21.15.131:0/50216949 learned_addr learned my addr 172.21.15.131:0/50216949 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.143+0000 7f9dc9e00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc06f730 0x7f9dcc1e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.143+0000 7f9dc3e00640 1 -- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dcc145590 msgr2=0x7f9dcc13f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.143+0000 7f9dca800640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dcc145590 0x7f9dcc13f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.143+0000 7f9dc3e00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dcc145590 0x7f9dcc13f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.143+0000 7f9dc3e00640 1 -- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc06f730 msgr2=0x7f9dcc1e4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.143+0000 7f9dc3e00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc06f730 0x7f9dcc1e4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.143+0000 7f9dc3e00640 1 -- 172.21.15.131:0/50216949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc13f590 con 0x7f9dcc0b3490 2024-04-28T12:38:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.143+0000 7f9dc9e00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc06f730 0x7f9dcc1e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.143+0000 7f9dca800640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dcc145590 0x7f9dcc13f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.143+0000 7f9dc3e00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dcc0b3490 0x7f9dcc140020 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9db400c4b0 tx=0x7f9db400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.144+0000 7f9dc2000640 1 -- 172.21.15.131:0/50216949 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4018070 con 0x7f9dcc0b3490 2024-04-28T12:38:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.144+0000 7f9dc2000640 1 -- 172.21.15.131:0/50216949 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db4007500 con 0x7f9dcc0b3490 2024-04-28T12:38:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.144+0000 7f9dd0c00640 1 -- 172.21.15.131:0/50216949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dcc1e6e40 con 0x7f9dcc0b3490 2024-04-28T12:38:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.144+0000 7f9dc2000640 1 -- 172.21.15.131:0/50216949 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4004a20 con 0x7f9dcc0b3490 2024-04-28T12:38:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.144+0000 7f9dd0c00640 1 -- 172.21.15.131:0/50216949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dcc13db60 con 0x7f9dcc0b3490 2024-04-28T12:38:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.145+0000 7f9dd0c00640 1 -- 172.21.15.131:0/50216949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dcc069350 con 0x7f9dcc0b3490 2024-04-28T12:38:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.146+0000 7f9dc2000640 1 -- 172.21.15.131:0/50216949 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9db401b070 con 0x7f9dcc0b3490 2024-04-28T12:38:52.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.147+0000 7f9dc2000640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9da806c8f0 0x7f9da806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:52.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.147+0000 7f9dc2000640 1 -- 172.21.15.131:0/50216949 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9db408b2c0 con 0x7f9dcc0b3490 2024-04-28T12:38:52.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.148+0000 7f9dc9e00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9da806c8f0 0x7f9da806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:52.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.149+0000 7f9dc9e00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9da806c8f0 0x7f9da806edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc034000 tx=0x7f9dbc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:52.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.152+0000 7f9dc2000640 1 -- 172.21.15.131:0/50216949 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9db4059310 con 0x7f9dcc0b3490 2024-04-28T12:38:52.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.361+0000 7f9dd0c00640 1 -- 172.21.15.131:0/50216949 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9dcc0d04d0 con 0x7f9dcc0b3490 2024-04-28T12:38:52.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.362+0000 7f9dc2000640 1 -- 172.21.15.131:0/50216949 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f9db4058c10 con 0x7f9dcc0b3490 2024-04-28T12:38:52.359 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-04-28T12:38:52.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.364+0000 7f9dd0c00640 1 -- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9da806c8f0 msgr2=0x7f9da806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:52.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.364+0000 7f9dd0c00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9da806c8f0 0x7f9da806edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc034000 tx=0x7f9dbc02e000 comp rx=0 tx=0).stop 2024-04-28T12:38:52.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.364+0000 7f9dd0c00640 1 -- 172.21.15.131:0/50216949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dcc0b3490 msgr2=0x7f9dcc140020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:52.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.364+0000 7f9dd0c00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dcc0b3490 0x7f9dcc140020 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9db400c4b0 tx=0x7f9db400c970 comp rx=0 tx=0).stop 2024-04-28T12:38:52.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.364+0000 7f9dd0c00640 1 -- 172.21.15.131:0/50216949 shutdown_connections 2024-04-28T12:38:52.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.364+0000 7f9dd0c00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9da806c8f0 0x7f9da806edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.364+0000 7f9dd0c00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dcc145590 0x7f9dcc13f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.364+0000 7f9dd0c00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9dcc0b3490 0x7f9dcc140020 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.364+0000 7f9dd0c00640 1 --2- 172.21.15.131:0/50216949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dcc06f730 0x7f9dcc1e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.365+0000 7f9dd0c00640 1 -- 172.21.15.131:0/50216949 >> 172.21.15.131:0/50216949 conn(0x7f9dcc0b2900 msgr2=0x7f9dcc0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:52.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.365+0000 7f9dd0c00640 1 -- 172.21.15.131:0/50216949 shutdown_connections 2024-04-28T12:38:52.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.365+0000 7f9dd0c00640 1 -- 172.21.15.131:0/50216949 wait complete. 2024-04-28T12:38:52.379 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi157.front.sepia.ceph.com... 2024-04-28T12:38:52.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:52.379 DEBUG:teuthology.orchestra.run.smithi157:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-28T12:38:52.468 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:38:52.468 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:38:52.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:38:52.468 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-28T12:38:52.517 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-28T12:38:52.517 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:38:52.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.726+0000 7f3a45e00640 1 Processor -- start 2024-04-28T12:38:52.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.727+0000 7f3a45e00640 1 -- start start 2024-04-28T12:38:52.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.728+0000 7f3a45e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f3a4015cab0 0x7f3a400afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:52.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.728+0000 7f3a45e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3a400b0020 0x7f3a400d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:52.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.728+0000 7f3a45e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a400b0650 con 0x7f3a400d1080 2024-04-28T12:38:52.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.728+0000 7f3a45e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f3a400d0900 con 0x7f3a400b0020 2024-04-28T12:38:52.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.728+0000 7f3a45e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f3a401635a0 con 0x7f3a4015cab0 2024-04-28T12:38:52.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.729+0000 7f3a36000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3a400b0020 0x7f3a400d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:52.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.729+0000 7f3a3f400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3a400d1080 0x7f3a400ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35098/0 (socket says 172.21.15.131:35098) 2024-04-28T12:38:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.729+0000 7f3a3f400640 1 -- 172.21.15.131:0/3816980748 learned_addr learned my addr 172.21.15.131:0/3816980748 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.729+0000 7f3a3ea00640 1 --2- 172.21.15.131:0/3816980748 >> v2:172.21.15.157:3300/0 conn(0x7f3a4015cab0 0x7f3a400afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.729+0000 7f3a36000640 1 -- 172.21.15.131:0/3816980748 >> v2:172.21.15.157:3300/0 conn(0x7f3a4015cab0 msgr2=0x7f3a400afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.729+0000 7f3a36000640 1 --2- 172.21.15.131:0/3816980748 >> v2:172.21.15.157:3300/0 conn(0x7f3a4015cab0 0x7f3a400afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.729+0000 7f3a36000640 1 -- 172.21.15.131:0/3816980748 >> v1:172.21.15.131:6789/0 conn(0x7f3a400d1080 legacy=0x7f3a400ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:52.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.729+0000 7f3a36000640 1 -- 172.21.15.131:0/3816980748 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a40163a30 con 0x7f3a400b0020 2024-04-28T12:38:52.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.729+0000 7f3a3ea00640 1 --2- 172.21.15.131:0/3816980748 >> v2:172.21.15.157:3300/0 conn(0x7f3a4015cab0 0x7f3a400afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:52.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.730+0000 7f3a36000640 1 --2- 172.21.15.131:0/3816980748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a400b0020 0x7f3a400d0b40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3a4006d120 tx=0x7f3a2802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38741433b7e6ba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:52.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.730+0000 7f3a3e000640 1 -- 172.21.15.131:0/3816980748 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a28037070 con 0x7f3a400b0020 2024-04-28T12:38:52.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.731+0000 7f3a3e000640 1 -- 172.21.15.131:0/3816980748 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a28032440 con 0x7f3a400b0020 2024-04-28T12:38:52.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.731+0000 7f3a3e000640 1 -- 172.21.15.131:0/3816980748 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a28032810 con 0x7f3a400b0020 2024-04-28T12:38:52.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.731+0000 7f3a45e00640 1 -- 172.21.15.131:0/3816980748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a400b0020 msgr2=0x7f3a400d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:52.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.731+0000 7f3a45e00640 1 --2- 172.21.15.131:0/3816980748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a400b0020 0x7f3a400d0b40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3a4006d120 tx=0x7f3a2802a1a0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.731+0000 7f3a45e00640 1 -- 172.21.15.131:0/3816980748 shutdown_connections 2024-04-28T12:38:52.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.731+0000 7f3a45e00640 1 --2- 172.21.15.131:0/3816980748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a400b0020 0x7f3a400d0b40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.731+0000 7f3a45e00640 1 --2- 172.21.15.131:0/3816980748 >> v2:172.21.15.157:3300/0 conn(0x7f3a4015cab0 0x7f3a400afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.731+0000 7f3a45e00640 1 -- 172.21.15.131:0/3816980748 >> 172.21.15.131:0/3816980748 conn(0x7f3a400b2900 msgr2=0x7f3a400b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:52.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.731+0000 7f3a45e00640 1 -- 172.21.15.131:0/3816980748 shutdown_connections 2024-04-28T12:38:52.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.731+0000 7f3a45e00640 1 -- 172.21.15.131:0/3816980748 wait complete. 2024-04-28T12:38:52.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.733+0000 7f3a45e00640 1 Processor -- start 2024-04-28T12:38:52.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.733+0000 7f3a45e00640 1 -- start start 2024-04-28T12:38:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.734+0000 7f3a45e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a400b0020 0x7f3a4014c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.734+0000 7f3a45e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400d1080 0x7f3a400b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.734+0000 7f3a45e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a4015cab0 0x7f3a400b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.734+0000 7f3a45e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3a40162f50 con 0x7f3a400b0020 2024-04-28T12:38:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.734+0000 7f3a45e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3a400d14b0 con 0x7f3a4015cab0 2024-04-28T12:38:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.734+0000 7f3a45e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a40163230 con 0x7f3a400d1080 2024-04-28T12:38:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.734+0000 7f3a3ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a400b0020 0x7f3a4014c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.734+0000 7f3a3ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a400b0020 0x7f3a4014c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41452/0 (socket says 172.21.15.131:41452) 2024-04-28T12:38:52.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.734+0000 7f3a3ea00640 1 -- 172.21.15.131:0/1810585249 learned_addr learned my addr 172.21.15.131:0/1810585249 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:52.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.735+0000 7f3a3ea00640 1 -- 172.21.15.131:0/1810585249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400d1080 msgr2=0x7f3a400b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:52.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.735+0000 7f3a3ea00640 1 --2- 172.21.15.131:0/1810585249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400d1080 0x7f3a400b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.735+0000 7f3a3ea00640 1 -- 172.21.15.131:0/1810585249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a4015cab0 msgr2=0x7f3a400b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:52.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.735+0000 7f3a3ea00640 1 --2- 172.21.15.131:0/1810585249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a4015cab0 0x7f3a400b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.735+0000 7f3a3ea00640 1 -- 172.21.15.131:0/1810585249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a40156fd0 con 0x7f3a400b0020 2024-04-28T12:38:52.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.735+0000 7f3a3ea00640 1 --2- 172.21.15.131:0/1810585249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a400b0020 0x7f3a4014c4d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000ce30 tx=0x7f3a3000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:52.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.735+0000 7f3a37e00640 1 -- 172.21.15.131:0/1810585249 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3000f890 con 0x7f3a400b0020 2024-04-28T12:38:52.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.735+0000 7f3a37e00640 1 -- 172.21.15.131:0/1810585249 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a3000fa30 con 0x7f3a400b0020 2024-04-28T12:38:52.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.735+0000 7f3a37e00640 1 -- 172.21.15.131:0/1810585249 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3000fe00 con 0x7f3a400b0020 2024-04-28T12:38:52.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.735+0000 7f3a45e00640 1 -- 172.21.15.131:0/1810585249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4015f4d0 con 0x7f3a400b0020 2024-04-28T12:38:52.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.736+0000 7f3a45e00640 1 -- 172.21.15.131:0/1810585249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4015fa90 con 0x7f3a400b0020 2024-04-28T12:38:52.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.737+0000 7f3a45e00640 1 -- 172.21.15.131:0/1810585249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a400afe40 con 0x7f3a400b0020 2024-04-28T12:38:52.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.738+0000 7f3a37e00640 1 -- 172.21.15.131:0/1810585249 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a30018400 con 0x7f3a400b0020 2024-04-28T12:38:52.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.739+0000 7f3a37e00640 1 --2- 172.21.15.131:0/1810585249 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3a1c06c8f0 0x7f3a1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:52.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.739+0000 7f3a37e00640 1 -- 172.21.15.131:0/1810585249 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3a3008cd20 con 0x7f3a400b0020 2024-04-28T12:38:52.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.740+0000 7f3a36000640 1 --2- 172.21.15.131:0/1810585249 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3a1c06c8f0 0x7f3a1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:52.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.741+0000 7f3a36000640 1 --2- 172.21.15.131:0/1810585249 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3a1c06c8f0 0x7f3a1c06edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3a28034000 tx=0x7f3a2802c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:52.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.744+0000 7f3a37e00640 1 -- 172.21.15.131:0/1810585249 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a400afe40 con 0x7f3a400b0020 2024-04-28T12:38:52.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.986+0000 7f3a45e00640 1 -- 172.21.15.131:0/1810585249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a400ce460 con 0x7f3a400b0020 2024-04-28T12:38:52.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.987+0000 7f3a37e00640 1 -- 172.21.15.131:0/1810585249 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5191 (secure 0 0 0) 0x7f3a40156fd0 con 0x7f3a400b0020 2024-04-28T12:38:52.984 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:52.984 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1064054651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1064054651},{"type":"v1","addr":"172.21.15.131:6834","nonce":1064054651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/398655652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":398655652},{"type":"v1","addr":"172.21.15.131:6835","nonce":398655652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2619661531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2619661531},{"type":"v1","addr":"172.21.15.157:6837","nonce":2619661531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:37:22.236351+0000","modified":"2024-04-28T12:37:50.533095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1937780695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1937780695},{"type":"v1","addr":"172.21.15.157:6835","nonce":1937780695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34160,"qdb_cluster":[34160]},"id":1}]} 2024-04-28T12:38:52.984 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-04-28T12:38:52.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.990+0000 7f3a45e00640 1 -- 172.21.15.131:0/1810585249 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3a1c06c8f0 msgr2=0x7f3a1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:52.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.990+0000 7f3a45e00640 1 --2- 172.21.15.131:0/1810585249 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3a1c06c8f0 0x7f3a1c06edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3a28034000 tx=0x7f3a2802c000 comp rx=0 tx=0).stop 2024-04-28T12:38:52.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.990+0000 7f3a45e00640 1 -- 172.21.15.131:0/1810585249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a400b0020 msgr2=0x7f3a4014c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:52.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.990+0000 7f3a45e00640 1 --2- 172.21.15.131:0/1810585249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a400b0020 0x7f3a4014c4d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000ce30 tx=0x7f3a3000a650 comp rx=0 tx=0).stop 2024-04-28T12:38:52.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.990+0000 7f3a45e00640 1 -- 172.21.15.131:0/1810585249 shutdown_connections 2024-04-28T12:38:52.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.990+0000 7f3a45e00640 1 --2- 172.21.15.131:0/1810585249 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3a1c06c8f0 0x7f3a1c06edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.990+0000 7f3a45e00640 1 --2- 172.21.15.131:0/1810585249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a4015cab0 0x7f3a400b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.990+0000 7f3a45e00640 1 --2- 172.21.15.131:0/1810585249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a400d1080 0x7f3a400b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.990+0000 7f3a45e00640 1 --2- 172.21.15.131:0/1810585249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a400b0020 0x7f3a4014c4d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:52.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.990+0000 7f3a45e00640 1 -- 172.21.15.131:0/1810585249 >> 172.21.15.131:0/1810585249 conn(0x7f3a400b2900 msgr2=0x7f3a4015df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:52.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.990+0000 7f3a45e00640 1 -- 172.21.15.131:0/1810585249 shutdown_connections 2024-04-28T12:38:52.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:52.990+0000 7f3a45e00640 1 -- 172.21.15.131:0/1810585249 wait complete. 2024-04-28T12:38:53.004 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:53.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.203+0000 7f4ca9e00640 1 Processor -- start 2024-04-28T12:38:53.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.203+0000 7f4ca9e00640 1 -- start start 2024-04-28T12:38:53.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.205+0000 7f4ca9e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4ca40d1620 0x7f4ca40d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.205+0000 7f4ca9e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ca40d3810 0x7f4ca40b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.205+0000 7f4ca9e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca40d0480 con 0x7f4ca40d1a00 2024-04-28T12:38:53.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.205+0000 7f4ca9e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca40d0ce0 con 0x7f4ca40d3810 2024-04-28T12:38:53.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.205+0000 7f4ca9e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca40d0720 con 0x7f4ca40d1620 2024-04-28T12:38:53.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.205+0000 7f4ca9400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4ca40d3810 0x7f4ca40b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:53.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.205+0000 7f4ca3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4ca40d1a00 0x7f4ca40b4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35102/0 (socket says 172.21.15.131:35102) 2024-04-28T12:38:53.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.205+0000 7f4ca3e00640 1 -- 172.21.15.131:0/1688604134 learned_addr learned my addr 172.21.15.131:0/1688604134 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:53.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.205+0000 7f4ca8a00640 1 --2- 172.21.15.131:0/1688604134 >> v2:172.21.15.157:3300/0 conn(0x7f4ca40d1620 0x7f4ca40d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:53.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.206+0000 7f4ca9400640 1 -- 172.21.15.131:0/1688604134 >> v2:172.21.15.157:3300/0 conn(0x7f4ca40d1620 msgr2=0x7f4ca40d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:53.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.206+0000 7f4ca9400640 1 --2- 172.21.15.131:0/1688604134 >> v2:172.21.15.157:3300/0 conn(0x7f4ca40d1620 0x7f4ca40d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.206+0000 7f4ca9400640 1 -- 172.21.15.131:0/1688604134 >> v1:172.21.15.131:6789/0 conn(0x7f4ca40d1a00 legacy=0x7f4ca40b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:53.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.206+0000 7f4ca9400640 1 -- 172.21.15.131:0/1688604134 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca40d0a00 con 0x7f4ca40d3810 2024-04-28T12:38:53.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.206+0000 7f4ca3400640 1 -- 172.21.15.131:0/1688604134 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2925272861 0 0) 0x7f4ca40d0480 con 0x7f4ca40d1a00 2024-04-28T12:38:53.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.206+0000 7f4ca8a00640 1 --2- 172.21.15.131:0/1688604134 >> v2:172.21.15.157:3300/0 conn(0x7f4ca40d1620 0x7f4ca40d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:53.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.207+0000 7f4ca9400640 1 --2- 172.21.15.131:0/1688604134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca40d3810 0x7f4ca40b5280 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4ca408d330 tx=0x7f4c9c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5fc849d52f8e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:53.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.207+0000 7f4ca3400640 1 -- 172.21.15.131:0/1688604134 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c007740 con 0x7f4ca40d3810 2024-04-28T12:38:53.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.207+0000 7f4ca3400640 1 -- 172.21.15.131:0/1688604134 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c9c0078e0 con 0x7f4ca40d3810 2024-04-28T12:38:53.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.207+0000 7f4ca3400640 1 -- 172.21.15.131:0/1688604134 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c007cb0 con 0x7f4ca40d3810 2024-04-28T12:38:53.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.207+0000 7f4ca9e00640 1 -- 172.21.15.131:0/1688604134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca40d3810 msgr2=0x7f4ca40b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:53.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.207+0000 7f4ca9e00640 1 --2- 172.21.15.131:0/1688604134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca40d3810 0x7f4ca40b5280 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4ca408d330 tx=0x7f4c9c02a5f0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.207+0000 7f4ca9e00640 1 -- 172.21.15.131:0/1688604134 shutdown_connections 2024-04-28T12:38:53.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.207+0000 7f4ca9e00640 1 --2- 172.21.15.131:0/1688604134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca40d3810 0x7f4ca40b5280 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.207+0000 7f4ca9e00640 1 --2- 172.21.15.131:0/1688604134 >> v2:172.21.15.157:3300/0 conn(0x7f4ca40d1620 0x7f4ca40d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.207+0000 7f4ca9e00640 1 -- 172.21.15.131:0/1688604134 >> 172.21.15.131:0/1688604134 conn(0x7f4ca40d4c30 msgr2=0x7f4ca40d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:53.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.208+0000 7f4ca9e00640 1 -- 172.21.15.131:0/1688604134 shutdown_connections 2024-04-28T12:38:53.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.208+0000 7f4ca9e00640 1 -- 172.21.15.131:0/1688604134 wait complete. 2024-04-28T12:38:53.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.209+0000 7f4ca9e00640 1 Processor -- start 2024-04-28T12:38:53.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.209+0000 7f4ca9e00640 1 -- start start 2024-04-28T12:38:53.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.210+0000 7f4ca9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca40d1620 0x7f4ca40c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.210+0000 7f4ca9e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca40d1a00 0x7f4ca41f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.210+0000 7f4ca9e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca40d3810 0x7f4ca40ba180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.210+0000 7f4ca9e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca40b57c0 con 0x7f4ca40d1620 2024-04-28T12:38:53.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.210+0000 7f4ca9e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca40d1e60 con 0x7f4ca40d3810 2024-04-28T12:38:53.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.210+0000 7f4ca9e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca40b5940 con 0x7f4ca40d1a00 2024-04-28T12:38:53.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.211+0000 7f4ca8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca40d1620 0x7f4ca40c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.211+0000 7f4ca8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca40d1620 0x7f4ca40c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41468/0 (socket says 172.21.15.131:41468) 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.211+0000 7f4ca8a00640 1 -- 172.21.15.131:0/471854598 learned_addr learned my addr 172.21.15.131:0/471854598 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.211+0000 7f4ca8a00640 1 -- 172.21.15.131:0/471854598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca40d1a00 msgr2=0x7f4ca41f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.211+0000 7f4ca8a00640 1 --2- 172.21.15.131:0/471854598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca40d1a00 0x7f4ca41f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.211+0000 7f4ca8a00640 1 -- 172.21.15.131:0/471854598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca40d3810 msgr2=0x7f4ca40ba180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.211+0000 7f4ca8a00640 1 --2- 172.21.15.131:0/471854598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca40d3810 0x7f4ca40ba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.211+0000 7f4ca8a00640 1 -- 172.21.15.131:0/471854598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca40cced0 con 0x7f4ca40d1620 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.211+0000 7f4ca8a00640 1 --2- 172.21.15.131:0/471854598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca40d1620 0x7f4ca40c8e90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c820 tx=0x7f4c9800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.211+0000 7f4ca0c00640 1 -- 172.21.15.131:0/471854598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98015070 con 0x7f4ca40d1620 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.211+0000 7f4ca0c00640 1 -- 172.21.15.131:0/471854598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c98007a30 con 0x7f4ca40d1620 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.211+0000 7f4ca9e00640 1 -- 172.21.15.131:0/471854598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca40cd1b0 con 0x7f4ca40d1620 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.212+0000 7f4ca0c00640 1 -- 172.21.15.131:0/471854598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98007e00 con 0x7f4ca40d1620 2024-04-28T12:38:53.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.212+0000 7f4ca9e00640 1 -- 172.21.15.131:0/471854598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca40ba6c0 con 0x7f4ca40d1620 2024-04-28T12:38:53.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.213+0000 7f4ca9e00640 1 -- 172.21.15.131:0/471854598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca4069350 con 0x7f4ca40d1620 2024-04-28T12:38:53.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.213+0000 7f4ca0c00640 1 -- 172.21.15.131:0/471854598 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c98018070 con 0x7f4ca40d1620 2024-04-28T12:38:53.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.214+0000 7f4ca0c00640 1 --2- 172.21.15.131:0/471854598 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4c8406c8f0 0x7f4c8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.214+0000 7f4ca0c00640 1 -- 172.21.15.131:0/471854598 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4c9808bfe0 con 0x7f4ca40d1620 2024-04-28T12:38:53.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.215+0000 7f4ca3e00640 1 --2- 172.21.15.131:0/471854598 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4c8406c8f0 0x7f4c8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:53.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.216+0000 7f4ca3e00640 1 --2- 172.21.15.131:0/471854598 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4c8406c8f0 0x7f4c8406edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4c90005880 tx=0x7f4c90007950 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:53.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.219+0000 7f4ca0c00640 1 -- 172.21.15.131:0/471854598 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c98059f20 con 0x7f4ca40d1620 2024-04-28T12:38:53.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.405+0000 7f4ca9e00640 1 -- 172.21.15.131:0/471854598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ca40b6eb0 con 0x7f4ca40d1620 2024-04-28T12:38:53.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.406+0000 7f4ca0c00640 1 -- 172.21.15.131:0/471854598 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7f4c98059820 con 0x7f4ca40d1620 2024-04-28T12:38:53.403 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:53.403 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:53.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.409+0000 7f4ca9e00640 1 -- 172.21.15.131:0/471854598 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4c8406c8f0 msgr2=0x7f4c8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:53.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.409+0000 7f4ca9e00640 1 --2- 172.21.15.131:0/471854598 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4c8406c8f0 0x7f4c8406edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4c90005880 tx=0x7f4c90007950 comp rx=0 tx=0).stop 2024-04-28T12:38:53.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.409+0000 7f4ca9e00640 1 -- 172.21.15.131:0/471854598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca40d1620 msgr2=0x7f4ca40c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:53.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.409+0000 7f4ca9e00640 1 --2- 172.21.15.131:0/471854598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca40d1620 0x7f4ca40c8e90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c820 tx=0x7f4c9800cc20 comp rx=0 tx=0).stop 2024-04-28T12:38:53.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.409+0000 7f4ca9e00640 1 -- 172.21.15.131:0/471854598 shutdown_connections 2024-04-28T12:38:53.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.409+0000 7f4ca9e00640 1 --2- 172.21.15.131:0/471854598 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4c8406c8f0 0x7f4c8406edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.409+0000 7f4ca9e00640 1 --2- 172.21.15.131:0/471854598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ca40d3810 0x7f4ca40ba180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.409+0000 7f4ca9e00640 1 --2- 172.21.15.131:0/471854598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ca40d1a00 0x7f4ca41f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.409+0000 7f4ca9e00640 1 --2- 172.21.15.131:0/471854598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca40d1620 0x7f4ca40c8e90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.409+0000 7f4ca9e00640 1 -- 172.21.15.131:0/471854598 >> 172.21.15.131:0/471854598 conn(0x7f4ca40d4c30 msgr2=0x7f4ca406d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:53.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.409+0000 7f4ca9e00640 1 -- 172.21.15.131:0/471854598 shutdown_connections 2024-04-28T12:38:53.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.409+0000 7f4ca9e00640 1 -- 172.21.15.131:0/471854598 wait complete. 2024-04-28T12:38:53.424 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:37:19.982605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:37:21.207222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:37:24.826402+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:38:53.425 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:38:53.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.622+0000 7fe4a2000640 1 Processor -- start 2024-04-28T12:38:53.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.622+0000 7fe4a2000640 1 -- start start 2024-04-28T12:38:53.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.623+0000 7fe4a2000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49c0b54b0 0x7fe49c1604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.623+0000 7fe4a2000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe49c06df40 0x7fe49c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.624+0000 7fe4a2000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49c0b3d80 con 0x7fe49c1458e0 2024-04-28T12:38:53.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.624+0000 7fe4a2000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe49c0b4060 con 0x7fe49c06df40 2024-04-28T12:38:53.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.624+0000 7fe4a2000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe49c0b4340 con 0x7fe49c0b54b0 2024-04-28T12:38:53.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.624+0000 7fe49b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe49c1458e0 0x7fe49c0b4da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35114/0 (socket says 172.21.15.131:35114) 2024-04-28T12:38:53.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.624+0000 7fe49b400640 1 -- 172.21.15.131:0/3325282146 learned_addr learned my addr 172.21.15.131:0/3325282146 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:53.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.624+0000 7fe493e00640 1 --2- 172.21.15.131:0/3325282146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49c0b54b0 0x7fe49c1604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:53.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.624+0000 7fe49be00640 1 --2- 172.21.15.131:0/3325282146 >> v2:172.21.15.157:3300/0 conn(0x7fe49c06df40 0x7fe49c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:53.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.624+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2351791337 0 0) 0x7fe49c0b3d80 con 0x7fe49c1458e0 2024-04-28T12:38:53.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.624+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe480003520 con 0x7fe49c1458e0 2024-04-28T12:38:53.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2725833327 0 0) 0x7fe480003520 con 0x7fe49c1458e0 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49c0b54b0 msgr2=0x7fe49c1604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 --2- 172.21.15.131:0/3325282146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49c0b54b0 0x7fe49c1604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 >> v2:172.21.15.157:3300/0 conn(0x7fe49c06df40 msgr2=0x7fe49c15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 --2- 172.21.15.131:0/3325282146 >> v2:172.21.15.157:3300/0 conn(0x7fe49c06df40 0x7fe49c15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c0b4620 con 0x7fe49c1458e0 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fe48c001670 con 0x7fe49c1458e0 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 >> v1:172.21.15.131:6789/0 conn(0x7fe49c1458e0 legacy=0x7fe49c0b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 --2- 172.21.15.131:0/3325282146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4800039b0 0x7fe480003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 --2- 172.21.15.131:0/3325282146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49c0b54b0 0x7fe480004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 --2- 172.21.15.131:0/3325282146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe480004d40 0x7fe4800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c0b4060 con 0x7fe480004d40 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c0b4340 con 0x7fe4800039b0 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe48000dbd0 con 0x7fe49c0b54b0 2024-04-28T12:38:53.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.625+0000 7fe49be00640 1 --2- 172.21.15.131:0/3325282146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe480004d40 0x7fe4800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:53.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.626+0000 7fe49be00640 1 -- 172.21.15.131:0/3325282146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49c0b54b0 msgr2=0x7fe480004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:53.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.626+0000 7fe49be00640 1 --2- 172.21.15.131:0/3325282146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe49c0b54b0 0x7fe480004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.626+0000 7fe49be00640 1 -- 172.21.15.131:0/3325282146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4800039b0 msgr2=0x7fe480003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:53.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.626+0000 7fe49b400640 1 --2- 172.21.15.131:0/3325282146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4800039b0 0x7fe480003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:53.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.626+0000 7fe49be00640 1 --2- 172.21.15.131:0/3325282146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4800039b0 0x7fe480003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.626+0000 7fe49be00640 1 -- 172.21.15.131:0/3325282146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe480003520 con 0x7fe480004d40 2024-04-28T12:38:53.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.627+0000 7fe49be00640 1 --2- 172.21.15.131:0/3325282146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe480004d40 0x7fe4800051c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe494009fc0 tx=0x7fe480005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d03de5bddfe4c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:53.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.627+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494009c10 con 0x7fe480004d40 2024-04-28T12:38:53.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.627+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe494036070 con 0x7fe480004d40 2024-04-28T12:38:53.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.627+0000 7fe49aa00640 1 -- 172.21.15.131:0/3325282146 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49402d070 con 0x7fe480004d40 2024-04-28T12:38:53.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.627+0000 7fe4a2000640 1 -- 172.21.15.131:0/3325282146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe480004d40 msgr2=0x7fe4800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:53.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.627+0000 7fe4a2000640 1 --2- 172.21.15.131:0/3325282146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe480004d40 0x7fe4800051c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe494009fc0 tx=0x7fe480005830 comp rx=0 tx=0).stop 2024-04-28T12:38:53.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.627+0000 7fe49be00640 1 -- 172.21.15.131:0/3325282146 reap_dead start 2024-04-28T12:38:53.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.628+0000 7fe4a2000640 1 -- 172.21.15.131:0/3325282146 shutdown_connections 2024-04-28T12:38:53.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.628+0000 7fe4a2000640 1 -- 172.21.15.131:0/3325282146 >> 172.21.15.131:0/3325282146 conn(0x7fe49c0b2900 msgr2=0x7fe49c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:53.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.628+0000 7fe4a2000640 1 -- 172.21.15.131:0/3325282146 shutdown_connections 2024-04-28T12:38:53.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.628+0000 7fe4a2000640 1 -- 172.21.15.131:0/3325282146 wait complete. 2024-04-28T12:38:53.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.630+0000 7fe4a2000640 1 Processor -- start 2024-04-28T12:38:53.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.630+0000 7fe4a2000640 1 -- start start 2024-04-28T12:38:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe4a2000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe480004d40 0x7fe49c1f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe4a2000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe49c0aec00 0x7fe49c15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe4a2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe49c0af060 0x7fe49c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe4a2000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c160650 con 0x7fe49c0af060 2024-04-28T12:38:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe4a2000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c1604d0 con 0x7fe49c0aec00 2024-04-28T12:38:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe4a2000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe49c1607d0 con 0x7fe480004d40 2024-04-28T12:38:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe49be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe49c0af060 0x7fe49c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe49be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe49c0af060 0x7fe49c15df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41474/0 (socket says 172.21.15.131:41474) 2024-04-28T12:38:53.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe49be00640 1 -- 172.21.15.131:0/2112354448 learned_addr learned my addr 172.21.15.131:0/2112354448 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe49b400640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe480004d40 0x7fe49c1f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe493e00640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe49c0aec00 0x7fe49c15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe49be00640 1 -- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe480004d40 msgr2=0x7fe49c1f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.631+0000 7fe49be00640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe480004d40 0x7fe49c1f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.632+0000 7fe49be00640 1 -- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe49c0aec00 msgr2=0x7fe49c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.632+0000 7fe49be00640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe49c0aec00 0x7fe49c15d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.632+0000 7fe49be00640 1 -- 172.21.15.131:0/2112354448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c0cf420 con 0x7fe49c0af060 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.632+0000 7fe493e00640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe49c0aec00 0x7fe49c15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.632+0000 7fe49be00640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe49c0af060 0x7fe49c15df20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe494033040 tx=0x7fe494009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.632+0000 7fe493400640 1 -- 172.21.15.131:0/2112354448 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494009c10 con 0x7fe49c0af060 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.632+0000 7fe493400640 1 -- 172.21.15.131:0/2112354448 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe494003b90 con 0x7fe49c0af060 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.632+0000 7fe493400640 1 -- 172.21.15.131:0/2112354448 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494004020 con 0x7fe49c0af060 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.632+0000 7fe4a2000640 1 -- 172.21.15.131:0/2112354448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c0cf700 con 0x7fe49c0af060 2024-04-28T12:38:53.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.632+0000 7fe4a2000640 1 -- 172.21.15.131:0/2112354448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c0cfcc0 con 0x7fe49c0af060 2024-04-28T12:38:53.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.634+0000 7fe4a2000640 1 -- 172.21.15.131:0/2112354448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49c069350 con 0x7fe49c0af060 2024-04-28T12:38:53.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.634+0000 7fe493400640 1 -- 172.21.15.131:0/2112354448 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe494005d10 con 0x7fe49c0af060 2024-04-28T12:38:53.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.635+0000 7fe493400640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe47806c8f0 0x7fe47806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:53.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.635+0000 7fe493400640 1 -- 172.21.15.131:0/2112354448 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe4940acfe0 con 0x7fe49c0af060 2024-04-28T12:38:53.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.636+0000 7fe49b400640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe47806c8f0 0x7fe47806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:53.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.637+0000 7fe49b400640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe47806c8f0 0x7fe47806edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe48c007970 tx=0x7fe48c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:53.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.639+0000 7fe493400640 1 -- 172.21.15.131:0/2112354448 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe49404c030 con 0x7fe49c0af060 2024-04-28T12:38:53.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.838+0000 7fe4a2000640 1 -- 172.21.15.131:0/2112354448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe49c06fac0 con 0x7fe49c0af060 2024-04-28T12:38:53.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.839+0000 7fe493400640 1 -- 172.21.15.131:0/2112354448 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fe49407af40 con 0x7fe49c0af060 2024-04-28T12:38:53.835 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:38:53.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.841+0000 7fe4a2000640 1 -- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe47806c8f0 msgr2=0x7fe47806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:53.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.841+0000 7fe4a2000640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe47806c8f0 0x7fe47806edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe48c007970 tx=0x7fe48c009040 comp rx=0 tx=0).stop 2024-04-28T12:38:53.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.841+0000 7fe4a2000640 1 -- 172.21.15.131:0/2112354448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe49c0af060 msgr2=0x7fe49c15df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:53.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.841+0000 7fe4a2000640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe49c0af060 0x7fe49c15df20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe494033040 tx=0x7fe494009750 comp rx=0 tx=0).stop 2024-04-28T12:38:53.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.842+0000 7fe4a2000640 1 -- 172.21.15.131:0/2112354448 shutdown_connections 2024-04-28T12:38:53.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.842+0000 7fe4a2000640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe47806c8f0 0x7fe47806edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.842+0000 7fe4a2000640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe49c0af060 0x7fe49c15df20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.842+0000 7fe4a2000640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe49c0aec00 0x7fe49c15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.842+0000 7fe4a2000640 1 --2- 172.21.15.131:0/2112354448 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe480004d40 0x7fe49c1f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:53.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.842+0000 7fe4a2000640 1 -- 172.21.15.131:0/2112354448 >> 172.21.15.131:0/2112354448 conn(0x7fe49c0b2900 msgr2=0x7fe49c15e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:53.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.842+0000 7fe4a2000640 1 -- 172.21.15.131:0/2112354448 shutdown_connections 2024-04-28T12:38:53.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:53.842+0000 7fe4a2000640 1 -- 172.21.15.131:0/2112354448 wait complete. 2024-04-28T12:38:53.857 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:38:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.066+0000 7fa5f3e00640 1 Processor -- start 2024-04-28T12:38:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.067+0000 7fa5f3e00640 1 -- start start 2024-04-28T12:38:54.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.068+0000 7fa5f3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f415f180 0x7fa5f40d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.068+0000 7fa5f3e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5f40d2d70 con 0x7fa5f40b3cb0 2024-04-28T12:38:54.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.068+0000 7fa5f3e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5f40d06e0 con 0x7fa5f40d3830 2024-04-28T12:38:54.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.068+0000 7fa5f3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f415f560 con 0x7fa5f415f180 2024-04-28T12:38:54.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.068+0000 7fa5f2000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa5f40b3cb0 0x7fa5f40d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35122/0 (socket says 172.21.15.131:35122) 2024-04-28T12:38:54.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.068+0000 7fa5f2000640 1 -- 172.21.15.131:0/3311656662 learned_addr learned my addr 172.21.15.131:0/3311656662 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:54.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f2a00640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f415f180 0x7fa5f40d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3564236139 0 0) 0x7fa5f40d2d70 con 0x7fa5f40b3cb0 2024-04-28T12:38:54.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5d4003520 con 0x7fa5f40b3cb0 2024-04-28T12:38:54.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3961133634 0 0) 0x7fa5d4003520 con 0x7fa5f40b3cb0 2024-04-28T12:38:54.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f415f180 msgr2=0x7fa5f40d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f1600640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f415f180 0x7fa5f40d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 >> v1:172.21.15.157:6789/0 conn(0x7fa5f40d3830 legacy=0x7fa5f40b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f40d09c0 con 0x7fa5f40b3cb0 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fa5dc0019f0 con 0x7fa5f40b3cb0 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f2a00640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f415f180 0x7fa5f40d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 >> v1:172.21.15.131:6789/0 conn(0x7fa5f40b3cb0 legacy=0x7fa5f40d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f1600640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5d4003a50 0x7fa5d4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f1600640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f415f180 0x7fa5d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.069+0000 7fa5f1600640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5d4004d90 0x7fa5d4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.070+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f415f560 con 0x7fa5d4004d90 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.070+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d400bb80 con 0x7fa5d4003a50 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.070+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa5d400bd80 con 0x7fa5f415f180 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.070+0000 7fa5f3400640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5d4004d90 0x7fa5d4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.070+0000 7fa5f2a00640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5d4003a50 0x7fa5d4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.070+0000 7fa5f2000640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f415f180 0x7fa5d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.070+0000 7fa5f3400640 1 -- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f415f180 msgr2=0x7fa5d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.070+0000 7fa5f3400640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f415f180 0x7fa5d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.070+0000 7fa5f3400640 1 -- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5d4003a50 msgr2=0x7fa5d4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.070+0000 7fa5f3400640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5d4003a50 0x7fa5d4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.070+0000 7fa5f3400640 1 -- 172.21.15.131:0/3311656662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d4003520 con 0x7fa5d4004d90 2024-04-28T12:38:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.070+0000 7fa5f2000640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f415f180 0x7fa5d4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.071+0000 7fa5f3400640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5d4004d90 0x7fa5d4005210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec001750 tx=0x7fa5d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85a3a7916042ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.072+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec001370 con 0x7fa5d4004d90 2024-04-28T12:38:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.072+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5ec0052d0 con 0x7fa5d4004d90 2024-04-28T12:38:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.072+0000 7fa5f1600640 1 -- 172.21.15.131:0/3311656662 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec02abe0 con 0x7fa5d4004d90 2024-04-28T12:38:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.072+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3311656662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5d4004d90 msgr2=0x7fa5d4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.072+0000 7fa5f3e00640 1 --2- 172.21.15.131:0/3311656662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5d4004d90 0x7fa5d4005210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec001750 tx=0x7fa5d4005830 comp rx=0 tx=0).stop 2024-04-28T12:38:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.072+0000 7fa5f3400640 1 -- 172.21.15.131:0/3311656662 reap_dead start 2024-04-28T12:38:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.072+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3311656662 shutdown_connections 2024-04-28T12:38:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.072+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3311656662 >> 172.21.15.131:0/3311656662 conn(0x7fa5f40d4bd0 msgr2=0x7fa5f40d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.072+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3311656662 shutdown_connections 2024-04-28T12:38:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.072+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3311656662 wait complete. 2024-04-28T12:38:54.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.074+0000 7fa5f3e00640 1 Processor -- start 2024-04-28T12:38:54.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.074+0000 7fa5f3e00640 1 -- start start 2024-04-28T12:38:54.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.075+0000 7fa5f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5d4004d90 0x7fa5f414e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.075+0000 7fa5f3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f40b5bf0 0x7fa5f40ba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.075+0000 7fa5f3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f41f8220 0x7fa5f41f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.075+0000 7fa5f3e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f40d11c0 con 0x7fa5d4004d90 2024-04-28T12:38:54.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.075+0000 7fa5f3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f40d2a40 con 0x7fa5f40b5bf0 2024-04-28T12:38:54.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.075+0000 7fa5f3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f40c8700 con 0x7fa5f41f8220 2024-04-28T12:38:54.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.076+0000 7fa5f2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5d4004d90 0x7fa5f414e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.076+0000 7fa5f2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5d4004d90 0x7fa5f414e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41500/0 (socket says 172.21.15.131:41500) 2024-04-28T12:38:54.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.076+0000 7fa5f2a00640 1 -- 172.21.15.131:0/3293778269 learned_addr learned my addr 172.21.15.131:0/3293778269 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:54.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.076+0000 7fa5f2a00640 1 -- 172.21.15.131:0/3293778269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f41f8220 msgr2=0x7fa5f41f86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.076+0000 7fa5f2a00640 1 --2- 172.21.15.131:0/3293778269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f41f8220 0x7fa5f41f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.076+0000 7fa5f2a00640 1 -- 172.21.15.131:0/3293778269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f40b5bf0 msgr2=0x7fa5f40ba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.076+0000 7fa5f2a00640 1 --2- 172.21.15.131:0/3293778269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f40b5bf0 0x7fa5f40ba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.076+0000 7fa5f2a00640 1 -- 172.21.15.131:0/3293778269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f40c8950 con 0x7fa5d4004d90 2024-04-28T12:38:54.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.076+0000 7fa5f2a00640 1 --2- 172.21.15.131:0/3293778269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5d4004d90 0x7fa5f414e090 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4010860 tx=0x7fa5e4010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:54.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.077+0000 7fa5eaa00640 1 -- 172.21.15.131:0/3293778269 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e401a070 con 0x7fa5d4004d90 2024-04-28T12:38:54.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.077+0000 7fa5eaa00640 1 -- 172.21.15.131:0/3293778269 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e400ca60 con 0x7fa5d4004d90 2024-04-28T12:38:54.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.077+0000 7fa5eaa00640 1 -- 172.21.15.131:0/3293778269 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400f040 con 0x7fa5d4004d90 2024-04-28T12:38:54.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.077+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3293778269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f40c8c30 con 0x7fa5d4004d90 2024-04-28T12:38:54.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.077+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3293778269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f40b50e0 con 0x7fa5d4004d90 2024-04-28T12:38:54.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.078+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3293778269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f40506c0 con 0x7fa5d4004d90 2024-04-28T12:38:54.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.079+0000 7fa5eaa00640 1 -- 172.21.15.131:0/3293778269 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5e4015470 con 0x7fa5d4004d90 2024-04-28T12:38:54.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.080+0000 7fa5eaa00640 1 --2- 172.21.15.131:0/3293778269 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa5d006c920 0x7fa5d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.080+0000 7fa5eaa00640 1 -- 172.21.15.131:0/3293778269 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa5e408c430 con 0x7fa5d4004d90 2024-04-28T12:38:54.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.080+0000 7fa5f2000640 1 --2- 172.21.15.131:0/3293778269 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa5d006c920 0x7fa5d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.082+0000 7fa5f2000640 1 --2- 172.21.15.131:0/3293778269 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa5d006c920 0x7fa5d006ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc007c40 tx=0x7fa5dc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:54.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.084+0000 7fa5eaa00640 1 -- 172.21.15.131:0/3293778269 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa5e405a450 con 0x7fa5d4004d90 2024-04-28T12:38:54.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.281+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3293778269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa5f40b1e30 con 0x7fa5d4004d90 2024-04-28T12:38:54.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.282+0000 7fa5eaa00640 1 -- 172.21.15.131:0/3293778269 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fa5e4059d50 con 0x7fa5d4004d90 2024-04-28T12:38:54.279 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:54.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.284+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3293778269 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa5d006c920 msgr2=0x7fa5d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.285+0000 7fa5f3e00640 1 --2- 172.21.15.131:0/3293778269 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa5d006c920 0x7fa5d006ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc007c40 tx=0x7fa5dc009040 comp rx=0 tx=0).stop 2024-04-28T12:38:54.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.285+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3293778269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5d4004d90 msgr2=0x7fa5f414e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.285+0000 7fa5f3e00640 1 --2- 172.21.15.131:0/3293778269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5d4004d90 0x7fa5f414e090 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4010860 tx=0x7fa5e4010d20 comp rx=0 tx=0).stop 2024-04-28T12:38:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.285+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3293778269 shutdown_connections 2024-04-28T12:38:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.285+0000 7fa5f3e00640 1 --2- 172.21.15.131:0/3293778269 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa5d006c920 0x7fa5d006ede0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.285+0000 7fa5f3e00640 1 --2- 172.21.15.131:0/3293778269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5f41f8220 0x7fa5f41f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.285+0000 7fa5f3e00640 1 --2- 172.21.15.131:0/3293778269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5f40b5bf0 0x7fa5f40ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.285+0000 7fa5f3e00640 1 --2- 172.21.15.131:0/3293778269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5d4004d90 0x7fa5f414e090 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.285+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3293778269 >> 172.21.15.131:0/3293778269 conn(0x7fa5f40d4bd0 msgr2=0x7fa5f415c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.285+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3293778269 shutdown_connections 2024-04-28T12:38:54.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.285+0000 7fa5f3e00640 1 -- 172.21.15.131:0/3293778269 wait complete. 2024-04-28T12:38:54.302 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:38:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.510+0000 7f29e5000640 1 Processor -- start 2024-04-28T12:38:54.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.511+0000 7f29e5000640 1 -- start start 2024-04-28T12:38:54.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.512+0000 7f29e5000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f29e015f180 0x7f29e00d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.512+0000 7f29e5000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f29e00d57d0 0x7f29e00d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.512+0000 7f29e5000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29e00cbff0 0x7f29e00cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.512+0000 7f29e5000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f29e00d1340 con 0x7f29e015f180 2024-04-28T12:38:54.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.512+0000 7f29e5000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f29e00d0490 con 0x7f29e00d57d0 2024-04-28T12:38:54.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.512+0000 7f29e5000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f29e00d0730 con 0x7f29e00cbff0 2024-04-28T12:38:54.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.512+0000 7f29df400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f29e015f180 0x7f29e00d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.512+0000 7f29df400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f29e015f180 0x7f29e00d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41512/0 (socket says 172.21.15.131:41512) 2024-04-28T12:38:54.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.513+0000 7f29df400640 1 -- 172.21.15.131:0/3487593271 learned_addr learned my addr 172.21.15.131:0/3487593271 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.513+0000 7f29d6a00640 1 --2- 172.21.15.131:0/3487593271 >> v2:172.21.15.157:3300/0 conn(0x7f29e00d57d0 0x7f29e00d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.513+0000 7f29dfe00640 1 --2- 172.21.15.131:0/3487593271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29e00cbff0 0x7f29e00cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.513+0000 7f29df400640 1 -- 172.21.15.131:0/3487593271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29e00cbff0 msgr2=0x7f29e00cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.513+0000 7f29df400640 1 --2- 172.21.15.131:0/3487593271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29e00cbff0 0x7f29e00cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.513+0000 7f29df400640 1 -- 172.21.15.131:0/3487593271 >> v2:172.21.15.157:3300/0 conn(0x7f29e00d57d0 msgr2=0x7f29e00d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.513+0000 7f29df400640 1 --2- 172.21.15.131:0/3487593271 >> v2:172.21.15.157:3300/0 conn(0x7f29e00d57d0 0x7f29e00d1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.513+0000 7f29df400640 1 -- 172.21.15.131:0/3487593271 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e00d0a10 con 0x7f29e015f180 2024-04-28T12:38:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.513+0000 7f29dfe00640 1 --2- 172.21.15.131:0/3487593271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29e00cbff0 0x7f29e00cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.514+0000 7f29df400640 1 --2- 172.21.15.131:0/3487593271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29e015f180 0x7f29e00d15c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f29e008eef0 tx=0x7f29d002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed0fad9b8c82d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.514+0000 7f29dea00640 1 -- 172.21.15.131:0/3487593271 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0037070 con 0x7f29e015f180 2024-04-28T12:38:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.514+0000 7f29dea00640 1 -- 172.21.15.131:0/3487593271 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d0032440 con 0x7f29e015f180 2024-04-28T12:38:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.514+0000 7f29dea00640 1 -- 172.21.15.131:0/3487593271 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0032810 con 0x7f29e015f180 2024-04-28T12:38:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.515+0000 7f29e5000640 1 -- 172.21.15.131:0/3487593271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29e015f180 msgr2=0x7f29e00d15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.515+0000 7f29e5000640 1 --2- 172.21.15.131:0/3487593271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29e015f180 0x7f29e00d15c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f29e008eef0 tx=0x7f29d002a140 comp rx=0 tx=0).stop 2024-04-28T12:38:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.515+0000 7f29e5000640 1 -- 172.21.15.131:0/3487593271 shutdown_connections 2024-04-28T12:38:54.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.515+0000 7f29e5000640 1 --2- 172.21.15.131:0/3487593271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29e00cbff0 0x7f29e00cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.515+0000 7f29e5000640 1 --2- 172.21.15.131:0/3487593271 >> v2:172.21.15.157:3300/0 conn(0x7f29e00d57d0 0x7f29e00d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.515+0000 7f29e5000640 1 --2- 172.21.15.131:0/3487593271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29e015f180 0x7f29e00d15c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.515+0000 7f29e5000640 1 -- 172.21.15.131:0/3487593271 >> 172.21.15.131:0/3487593271 conn(0x7f29e00d4bb0 msgr2=0x7f29e00d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.515+0000 7f29e5000640 1 -- 172.21.15.131:0/3487593271 shutdown_connections 2024-04-28T12:38:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.515+0000 7f29e5000640 1 -- 172.21.15.131:0/3487593271 wait complete. 2024-04-28T12:38:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.517+0000 7f29e5000640 1 Processor -- start 2024-04-28T12:38:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.518+0000 7f29e5000640 1 -- start start 2024-04-28T12:38:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.519+0000 7f29e5000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29e00cbff0 0x7f29e00cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.519+0000 7f29e5000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e00d57d0 0x7f29e015b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.519+0000 7f29e5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29e015f180 0x7f29e006ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.519+0000 7f29e5000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f29e00cc9a0 con 0x7f29e015f180 2024-04-28T12:38:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.519+0000 7f29e5000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f29e00d0fd0 con 0x7f29e00d57d0 2024-04-28T12:38:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.519+0000 7f29e5000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f29e00ccb20 con 0x7f29e00cbff0 2024-04-28T12:38:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.519+0000 7f29dfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29e015f180 0x7f29e006ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.519+0000 7f29dfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29e015f180 0x7f29e006ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41518/0 (socket says 172.21.15.131:41518) 2024-04-28T12:38:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.519+0000 7f29dfe00640 1 -- 172.21.15.131:0/2893736829 learned_addr learned my addr 172.21.15.131:0/2893736829 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.519+0000 7f29d6a00640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e00d57d0 0x7f29e015b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.520+0000 7f29df400640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29e00cbff0 0x7f29e00cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.520+0000 7f29dfe00640 1 -- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29e00cbff0 msgr2=0x7f29e00cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.520+0000 7f29dfe00640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29e00cbff0 0x7f29e00cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.520+0000 7f29dfe00640 1 -- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e00d57d0 msgr2=0x7f29e015b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.520+0000 7f29dfe00640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e00d57d0 0x7f29e015b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.520+0000 7f29dfe00640 1 -- 172.21.15.131:0/2893736829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e01f5450 con 0x7f29e015f180 2024-04-28T12:38:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.520+0000 7f29dfe00640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29e015f180 0x7f29e006ca50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f29d800cef0 tx=0x7f29d800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:54.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.520+0000 7f29d7e00640 1 -- 172.21.15.131:0/2893736829 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d801a070 con 0x7f29e015f180 2024-04-28T12:38:54.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.520+0000 7f29d7e00640 1 -- 172.21.15.131:0/2893736829 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d800d840 con 0x7f29e015f180 2024-04-28T12:38:54.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.520+0000 7f29e5000640 1 -- 172.21.15.131:0/2893736829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e00cab70 con 0x7f29e015f180 2024-04-28T12:38:54.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.520+0000 7f29d7e00640 1 -- 172.21.15.131:0/2893736829 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d800dc30 con 0x7f29e015f180 2024-04-28T12:38:54.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.520+0000 7f29e5000640 1 -- 172.21.15.131:0/2893736829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e015b560 con 0x7f29e015f180 2024-04-28T12:38:54.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.522+0000 7f29e5000640 1 -- 172.21.15.131:0/2893736829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e00caf60 con 0x7f29e015f180 2024-04-28T12:38:54.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.523+0000 7f29d7e00640 1 -- 172.21.15.131:0/2893736829 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29d801d070 con 0x7f29e015f180 2024-04-28T12:38:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.523+0000 7f29d7e00640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f29bc06c8f0 0x7f29bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.524+0000 7f29d7e00640 1 -- 172.21.15.131:0/2893736829 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f29d808c730 con 0x7f29e015f180 2024-04-28T12:38:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.524+0000 7f29df400640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f29bc06c8f0 0x7f29bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.525+0000 7f29df400640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f29bc06c8f0 0x7f29bc06edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f29d0034000 tx=0x7f29d002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:54.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.528+0000 7f29d7e00640 1 -- 172.21.15.131:0/2893736829 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f29d805a760 con 0x7f29e015f180 2024-04-28T12:38:54.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.728+0000 7f29e5000640 1 -- 172.21.15.131:0/2893736829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f29e00cad40 con 0x7f29e015f180 2024-04-28T12:38:54.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.729+0000 7f29d7e00640 1 -- 172.21.15.131:0/2893736829 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f29d805a060 con 0x7f29e015f180 2024-04-28T12:38:54.726 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:38:54.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.732+0000 7f29e5000640 1 -- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f29bc06c8f0 msgr2=0x7f29bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.732+0000 7f29e5000640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f29bc06c8f0 0x7f29bc06edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f29d0034000 tx=0x7f29d002e000 comp rx=0 tx=0).stop 2024-04-28T12:38:54.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.732+0000 7f29e5000640 1 -- 172.21.15.131:0/2893736829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29e015f180 msgr2=0x7f29e006ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.732+0000 7f29e5000640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29e015f180 0x7f29e006ca50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f29d800cef0 tx=0x7f29d800a710 comp rx=0 tx=0).stop 2024-04-28T12:38:54.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.732+0000 7f29e5000640 1 -- 172.21.15.131:0/2893736829 shutdown_connections 2024-04-28T12:38:54.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.732+0000 7f29e5000640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f29bc06c8f0 0x7f29bc06edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.732+0000 7f29e5000640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29e015f180 0x7f29e006ca50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.732+0000 7f29e5000640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29e00d57d0 0x7f29e015b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.732+0000 7f29e5000640 1 --2- 172.21.15.131:0/2893736829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29e00cbff0 0x7f29e00cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.732+0000 7f29e5000640 1 -- 172.21.15.131:0/2893736829 >> 172.21.15.131:0/2893736829 conn(0x7f29e00d4bb0 msgr2=0x7f29e015e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:54.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.733+0000 7f29e5000640 1 -- 172.21.15.131:0/2893736829 shutdown_connections 2024-04-28T12:38:54.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.733+0000 7f29e5000640 1 -- 172.21.15.131:0/2893736829 wait complete. 2024-04-28T12:38:54.749 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:38:54.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.962+0000 7f837ec00640 1 Processor -- start 2024-04-28T12:38:54.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.963+0000 7f837ec00640 1 -- start start 2024-04-28T12:38:54.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.964+0000 7f837ec00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f837815a7e0 0x7f83780b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.964+0000 7f837ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780b5410 0x7f83780b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.964+0000 7f837ec00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83780b3900 con 0x7f837815d2b0 2024-04-28T12:38:54.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.964+0000 7f837ec00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f837815abc0 con 0x7f837815a7e0 2024-04-28T12:38:54.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.964+0000 7f837ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83780b3b70 con 0x7f83780b5410 2024-04-28T12:38:54.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.965+0000 7f837d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f837815a7e0 0x7f83780b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.965+0000 7f837d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f837815a7e0 0x7f83780b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41534/0 (socket says 172.21.15.131:41534) 2024-04-28T12:38:54.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.965+0000 7f837d800640 1 -- 172.21.15.131:0/3489683612 learned_addr learned my addr 172.21.15.131:0/3489683612 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:54.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.965+0000 7f837e200640 1 --1- 172.21.15.131:0/3489683612 >> v1:172.21.15.131:6789/0 conn(0x7f837815d2b0 0x7f837815d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35128/0 (socket says 172.21.15.131:35128) 2024-04-28T12:38:54.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.965+0000 7f837d800640 1 -- 172.21.15.131:0/3489683612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780b5410 msgr2=0x7f83780b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:54.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.965+0000 7f837d800640 1 --2- 172.21.15.131:0/3489683612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780b5410 0x7f83780b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.965+0000 7f837d800640 1 -- 172.21.15.131:0/3489683612 >> v1:172.21.15.131:6789/0 conn(0x7f837815d2b0 legacy=0x7f837815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.965+0000 7f837d800640 1 -- 172.21.15.131:0/3489683612 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83780b3e50 con 0x7f837815a7e0 2024-04-28T12:38:54.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.966+0000 7f837d800640 1 --2- 172.21.15.131:0/3489683612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837815a7e0 0x7f83780b4650 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f836c009870 tx=0x7f836c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535c83306fe3ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:54.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.966+0000 7f8377e00640 1 -- 172.21.15.131:0/3489683612 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c037070 con 0x7f837815a7e0 2024-04-28T12:38:54.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.967+0000 7f8377e00640 1 -- 172.21.15.131:0/3489683612 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836c032440 con 0x7f837815a7e0 2024-04-28T12:38:54.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.967+0000 7f8377e00640 1 -- 172.21.15.131:0/3489683612 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c032810 con 0x7f837815a7e0 2024-04-28T12:38:54.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.967+0000 7f837ec00640 1 -- 172.21.15.131:0/3489683612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837815a7e0 msgr2=0x7f83780b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.967+0000 7f837ec00640 1 --2- 172.21.15.131:0/3489683612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837815a7e0 0x7f83780b4650 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f836c009870 tx=0x7f836c02a250 comp rx=0 tx=0).stop 2024-04-28T12:38:54.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.967+0000 7f837ec00640 1 -- 172.21.15.131:0/3489683612 shutdown_connections 2024-04-28T12:38:54.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.967+0000 7f837ec00640 1 --2- 172.21.15.131:0/3489683612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780b5410 0x7f83780b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.967+0000 7f837ec00640 1 --2- 172.21.15.131:0/3489683612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837815a7e0 0x7f83780b4650 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.967+0000 7f837ec00640 1 -- 172.21.15.131:0/3489683612 >> 172.21.15.131:0/3489683612 conn(0x7f8378068090 msgr2=0x7f83780684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:54.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.967+0000 7f837ec00640 1 -- 172.21.15.131:0/3489683612 shutdown_connections 2024-04-28T12:38:54.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.967+0000 7f837ec00640 1 -- 172.21.15.131:0/3489683612 wait complete. 2024-04-28T12:38:54.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.969+0000 7f837ec00640 1 Processor -- start 2024-04-28T12:38:54.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.969+0000 7f837ec00640 1 -- start start 2024-04-28T12:38:54.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.970+0000 7f837ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780b5410 0x7f8378150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.970+0000 7f837ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837815a7e0 0x7f8378071b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.970+0000 7f837ec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837815d2b0 0x7f83780ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.970+0000 7f837ec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f83780b4b90 con 0x7f837815d2b0 2024-04-28T12:38:54.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.970+0000 7f837ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f837815e010 con 0x7f837815a7e0 2024-04-28T12:38:54.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.970+0000 7f837ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f837815e2e0 con 0x7f83780b5410 2024-04-28T12:38:54.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.970+0000 7f837e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837815d2b0 0x7f83780ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.971+0000 7f837e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837815d2b0 0x7f83780ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41536/0 (socket says 172.21.15.131:41536) 2024-04-28T12:38:54.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.971+0000 7f837e200640 1 -- 172.21.15.131:0/1486764733 learned_addr learned my addr 172.21.15.131:0/1486764733 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:54.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.971+0000 7f837d800640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780b5410 0x7f8378150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.971+0000 7f837ce00640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837815a7e0 0x7f8378071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.971+0000 7f837e200640 1 -- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780b5410 msgr2=0x7f8378150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.971+0000 7f837e200640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780b5410 0x7f8378150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.971+0000 7f837e200640 1 -- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837815a7e0 msgr2=0x7f8378071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:54.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.971+0000 7f837e200640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837815a7e0 0x7f8378071b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:54.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.971+0000 7f837e200640 1 -- 172.21.15.131:0/1486764733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83781579c0 con 0x7f837815d2b0 2024-04-28T12:38:54.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.971+0000 7f837d800640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780b5410 0x7f8378150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:54.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.971+0000 7f837e200640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837815d2b0 0x7f83780ae8f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f837000c760 tx=0x7f837000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:54.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.972+0000 7f8375600640 1 -- 172.21.15.131:0/1486764733 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370017070 con 0x7f837815d2b0 2024-04-28T12:38:54.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.972+0000 7f8375600640 1 -- 172.21.15.131:0/1486764733 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f837000a640 con 0x7f837815d2b0 2024-04-28T12:38:54.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.972+0000 7f8375600640 1 -- 172.21.15.131:0/1486764733 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837000a9f0 con 0x7f837815d2b0 2024-04-28T12:38:54.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.972+0000 7f837ec00640 1 -- 172.21.15.131:0/1486764733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8378157ca0 con 0x7f837815d2b0 2024-04-28T12:38:54.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.972+0000 7f837ec00640 1 -- 172.21.15.131:0/1486764733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8378158260 con 0x7f837815d2b0 2024-04-28T12:38:54.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.973+0000 7f837ec00640 1 -- 172.21.15.131:0/1486764733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83780506c0 con 0x7f837815d2b0 2024-04-28T12:38:54.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.974+0000 7f8375600640 1 -- 172.21.15.131:0/1486764733 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f837001a070 con 0x7f837815d2b0 2024-04-28T12:38:54.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.975+0000 7f8375600640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f835806c8f0 0x7f835806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:54.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.976+0000 7f8375600640 1 -- 172.21.15.131:0/1486764733 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f837008b420 con 0x7f837815d2b0 2024-04-28T12:38:54.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.976+0000 7f837d800640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f835806c8f0 0x7f835806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:54.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.977+0000 7f837d800640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f835806c8f0 0x7f835806edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f836c034000 tx=0x7f836c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:54.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:54.980+0000 7f8375600640 1 -- 172.21.15.131:0/1486764733 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8370058e00 con 0x7f837815d2b0 2024-04-28T12:38:55.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.181+0000 7f837ec00640 1 -- 172.21.15.131:0/1486764733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f83780b5930 con 0x7f837815d2b0 2024-04-28T12:38:55.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.182+0000 7f8375600640 1 -- 172.21.15.131:0/1486764733 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f837005d100 con 0x7f837815d2b0 2024-04-28T12:38:55.179 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:38:55.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.185+0000 7f837ec00640 1 -- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f835806c8f0 msgr2=0x7f835806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.185+0000 7f837ec00640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f835806c8f0 0x7f835806edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f836c034000 tx=0x7f836c02e000 comp rx=0 tx=0).stop 2024-04-28T12:38:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.185+0000 7f837ec00640 1 -- 172.21.15.131:0/1486764733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837815d2b0 msgr2=0x7f83780ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.185+0000 7f837ec00640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837815d2b0 0x7f83780ae8f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f837000c760 tx=0x7f837000cc20 comp rx=0 tx=0).stop 2024-04-28T12:38:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.185+0000 7f837ec00640 1 -- 172.21.15.131:0/1486764733 shutdown_connections 2024-04-28T12:38:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.185+0000 7f837ec00640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f835806c8f0 0x7f835806edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.185+0000 7f837ec00640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f837815d2b0 0x7f83780ae8f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.185+0000 7f837ec00640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f837815a7e0 0x7f8378071b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.185+0000 7f837ec00640 1 --2- 172.21.15.131:0/1486764733 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83780b5410 0x7f8378150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.185+0000 7f837ec00640 1 -- 172.21.15.131:0/1486764733 >> 172.21.15.131:0/1486764733 conn(0x7f8378068090 msgr2=0x7f837815c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.185+0000 7f837ec00640 1 -- 172.21.15.131:0/1486764733 shutdown_connections 2024-04-28T12:38:55.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.185+0000 7f837ec00640 1 -- 172.21.15.131:0/1486764733 wait complete. 2024-04-28T12:38:55.203 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:38:55.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.407+0000 7f02ce800640 1 Processor -- start 2024-04-28T12:38:55.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.407+0000 7f02ce800640 1 -- start start 2024-04-28T12:38:55.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.409+0000 7f02ce800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f02c815f110 0x7f02c80d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.409+0000 7f02ce800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0760 0x7f02c80d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.409+0000 7f02ce800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c80d2c90 con 0x7f02c80d1610 2024-04-28T12:38:55.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.409+0000 7f02ce800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f02c815f4f0 con 0x7f02c815f110 2024-04-28T12:38:55.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.409+0000 7f02ce800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f02c80d1f90 con 0x7f02c80d0760 2024-04-28T12:38:55.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.409+0000 7f02cca00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f02c80d1610 0x7f02c80b4980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35134/0 (socket says 172.21.15.131:35134) 2024-04-28T12:38:55.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.409+0000 7f02cca00640 1 -- 172.21.15.131:0/3547886883 learned_addr learned my addr 172.21.15.131:0/3547886883 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:55.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02cd400640 1 --2- 172.21.15.131:0/3547886883 >> v2:172.21.15.157:3300/0 conn(0x7f02c815f110 0x7f02c80d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.409+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3418317638 0 0) 0x7f02c80d2c90 con 0x7f02c80d1610 2024-04-28T12:38:55.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02cde00640 1 --2- 172.21.15.131:0/3547886883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0760 0x7f02c80d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02b0003520 con 0x7f02c80d1610 2024-04-28T12:38:55.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 78708669 0 0) 0x7f02b0003520 con 0x7f02c80d1610 2024-04-28T12:38:55.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0760 msgr2=0x7f02c80d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:55.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 --2- 172.21.15.131:0/3547886883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0760 0x7f02c80d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 >> v2:172.21.15.157:3300/0 conn(0x7f02c815f110 msgr2=0x7f02c80d38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:55.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 --2- 172.21.15.131:0/3547886883 >> v2:172.21.15.157:3300/0 conn(0x7f02c815f110 0x7f02c80d38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c80d29b0 con 0x7f02c80d1610 2024-04-28T12:38:55.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f02b4001670 con 0x7f02c80d1610 2024-04-28T12:38:55.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02cd400640 1 --2- 172.21.15.131:0/3547886883 >> v2:172.21.15.157:3300/0 conn(0x7f02c815f110 0x7f02c80d38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:38:55.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 >> v1:172.21.15.131:6789/0 conn(0x7f02c80d1610 legacy=0x7f02c80b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:55.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 --2- 172.21.15.131:0/3547886883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b00039b0 0x7f02b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 --2- 172.21.15.131:0/3547886883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0760 0x7f02b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 --2- 172.21.15.131:0/3547886883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02b0004d40 0x7f02b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f02c815f4f0 con 0x7f02b00039b0 2024-04-28T12:38:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f02c80d1f90 con 0x7f02b0004d40 2024-04-28T12:38:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f02b000ba70 con 0x7f02c80d0760 2024-04-28T12:38:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.410+0000 7f02cd400640 1 --2- 172.21.15.131:0/3547886883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b00039b0 0x7f02b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.411+0000 7f02cca00640 1 --2- 172.21.15.131:0/3547886883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0760 0x7f02b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.411+0000 7f02cd400640 1 -- 172.21.15.131:0/3547886883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0760 msgr2=0x7f02b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.411+0000 7f02cd400640 1 --2- 172.21.15.131:0/3547886883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02c80d0760 0x7f02b0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.411+0000 7f02cd400640 1 -- 172.21.15.131:0/3547886883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02b0004d40 msgr2=0x7f02b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.411+0000 7f02cd400640 1 --2- 172.21.15.131:0/3547886883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02b0004d40 0x7f02b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.411+0000 7f02cd400640 1 -- 172.21.15.131:0/3547886883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b0003520 con 0x7f02b00039b0 2024-04-28T12:38:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.412+0000 7f02cd400640 1 --2- 172.21.15.131:0/3547886883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b00039b0 0x7f02b0003dd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0098d0 tx=0x7f02b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7924b5d8cdaeaf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.412+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00f040 con 0x7f02b00039b0 2024-04-28T12:38:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.412+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02bc02ad50 con 0x7f02b00039b0 2024-04-28T12:38:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.412+0000 7f02c7e00640 1 -- 172.21.15.131:0/3547886883 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc004250 con 0x7f02b00039b0 2024-04-28T12:38:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.413+0000 7f02ce800640 1 -- 172.21.15.131:0/3547886883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b00039b0 msgr2=0x7f02b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.413+0000 7f02ce800640 1 --2- 172.21.15.131:0/3547886883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02b00039b0 0x7f02b0003dd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0098d0 tx=0x7f02b0005830 comp rx=0 tx=0).stop 2024-04-28T12:38:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.413+0000 7f02cde00640 1 -- 172.21.15.131:0/3547886883 reap_dead start 2024-04-28T12:38:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.413+0000 7f02ce800640 1 -- 172.21.15.131:0/3547886883 shutdown_connections 2024-04-28T12:38:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.413+0000 7f02ce800640 1 -- 172.21.15.131:0/3547886883 >> 172.21.15.131:0/3547886883 conn(0x7f02c80d4c40 msgr2=0x7f02c80d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.413+0000 7f02ce800640 1 -- 172.21.15.131:0/3547886883 shutdown_connections 2024-04-28T12:38:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.413+0000 7f02ce800640 1 -- 172.21.15.131:0/3547886883 wait complete. 2024-04-28T12:38:55.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.415+0000 7f02ce800640 1 Processor -- start 2024-04-28T12:38:55.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.415+0000 7f02ce800640 1 -- start start 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.416+0000 7f02ce800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02b00039b0 0x7f02c81f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.416+0000 7f02ce800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c80c9df0 0x7f02c814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.416+0000 7f02ce800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c806d3f0 0x7f02c806d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.416+0000 7f02ce800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f02c80d5760 con 0x7f02c80c9df0 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.416+0000 7f02ce800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f02c80b4460 con 0x7f02c806d3f0 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.416+0000 7f02ce800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f02c80d58e0 con 0x7f02b00039b0 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.416+0000 7f02cca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c80c9df0 0x7f02c814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.416+0000 7f02cca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c80c9df0 0x7f02c814ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41554/0 (socket says 172.21.15.131:41554) 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.416+0000 7f02cca00640 1 -- 172.21.15.131:0/110616675 learned_addr learned my addr 172.21.15.131:0/110616675 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.416+0000 7f02cd400640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02b00039b0 0x7f02c81f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.416+0000 7f02cde00640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c806d3f0 0x7f02c806d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.417+0000 7f02cca00640 1 -- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02b00039b0 msgr2=0x7f02c81f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:55.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.417+0000 7f02cca00640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02b00039b0 0x7f02c81f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.417+0000 7f02cca00640 1 -- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c806d3f0 msgr2=0x7f02c806d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.417+0000 7f02cca00640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c806d3f0 0x7f02c806d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.417+0000 7f02cca00640 1 -- 172.21.15.131:0/110616675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c80cb570 con 0x7f02c80c9df0 2024-04-28T12:38:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.417+0000 7f02cd400640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02b00039b0 0x7f02c81f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.417+0000 7f02cca00640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c80c9df0 0x7f02c814ff60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f02b400d810 tx=0x7f02b400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.417+0000 7f02c5600640 1 -- 172.21.15.131:0/110616675 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4019070 con 0x7f02c80c9df0 2024-04-28T12:38:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.417+0000 7f02c5600640 1 -- 172.21.15.131:0/110616675 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b40074e0 con 0x7f02c80c9df0 2024-04-28T12:38:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.417+0000 7f02ce800640 1 -- 172.21.15.131:0/110616675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c80ca680 con 0x7f02c80c9df0 2024-04-28T12:38:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.417+0000 7f02c5600640 1 -- 172.21.15.131:0/110616675 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b40049e0 con 0x7f02c80c9df0 2024-04-28T12:38:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.417+0000 7f02ce800640 1 -- 172.21.15.131:0/110616675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c80cac40 con 0x7f02c80c9df0 2024-04-28T12:38:55.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.418+0000 7f02ce800640 1 -- 172.21.15.131:0/110616675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c8069350 con 0x7f02c80c9df0 2024-04-28T12:38:55.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.419+0000 7f02c5600640 1 -- 172.21.15.131:0/110616675 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02b401c070 con 0x7f02c80c9df0 2024-04-28T12:38:55.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.420+0000 7f02c5600640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f02a806c8f0 0x7f02a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.420+0000 7f02c5600640 1 -- 172.21.15.131:0/110616675 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f02b408bae0 con 0x7f02c80c9df0 2024-04-28T12:38:55.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.421+0000 7f02cd400640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f02a806c8f0 0x7f02a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.422+0000 7f02cd400640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f02a806c8f0 0x7f02a806edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f02bc033000 tx=0x7f02bc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:55.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.425+0000 7f02c5600640 1 -- 172.21.15.131:0/110616675 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f02b4059a70 con 0x7f02c80c9df0 2024-04-28T12:38:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.669+0000 7f02ce800640 1 -- 172.21.15.131:0/110616675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02c80d2c90 con 0x7f02c80c9df0 2024-04-28T12:38:55.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.671+0000 7f02c5600640 1 -- 172.21.15.131:0/110616675 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5191 (secure 0 0 0) 0x7f02b4059370 con 0x7f02c80c9df0 2024-04-28T12:38:55.667 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:55.668 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1064054651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1064054651},{"type":"v1","addr":"172.21.15.131:6834","nonce":1064054651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/398655652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":398655652},{"type":"v1","addr":"172.21.15.131:6835","nonce":398655652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2619661531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2619661531},{"type":"v1","addr":"172.21.15.157:6837","nonce":2619661531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:37:22.236351+0000","modified":"2024-04-28T12:37:50.533095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1937780695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1937780695},{"type":"v1","addr":"172.21.15.157:6835","nonce":1937780695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34160,"qdb_cluster":[34160]},"id":1}]} 2024-04-28T12:38:55.668 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-04-28T12:38:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.673+0000 7f02ce800640 1 -- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f02a806c8f0 msgr2=0x7f02a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.673+0000 7f02ce800640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f02a806c8f0 0x7f02a806edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f02bc033000 tx=0x7f02bc02e000 comp rx=0 tx=0).stop 2024-04-28T12:38:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.673+0000 7f02ce800640 1 -- 172.21.15.131:0/110616675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c80c9df0 msgr2=0x7f02c814ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.673+0000 7f02ce800640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c80c9df0 0x7f02c814ff60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f02b400d810 tx=0x7f02b400dcd0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.673+0000 7f02ce800640 1 -- 172.21.15.131:0/110616675 shutdown_connections 2024-04-28T12:38:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.673+0000 7f02ce800640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f02a806c8f0 0x7f02a806edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.673+0000 7f02ce800640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02c806d3f0 0x7f02c806d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.673+0000 7f02ce800640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02c80c9df0 0x7f02c814ff60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.673+0000 7f02ce800640 1 --2- 172.21.15.131:0/110616675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02b00039b0 0x7f02c81f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.673+0000 7f02ce800640 1 -- 172.21.15.131:0/110616675 >> 172.21.15.131:0/110616675 conn(0x7f02c80d4c40 msgr2=0x7f02c80b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:55.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.674+0000 7f02ce800640 1 -- 172.21.15.131:0/110616675 shutdown_connections 2024-04-28T12:38:55.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.674+0000 7f02ce800640 1 -- 172.21.15.131:0/110616675 wait complete. 2024-04-28T12:38:55.690 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:55.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.879+0000 7f2667a00640 1 Processor -- start 2024-04-28T12:38:55.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.879+0000 7f2667a00640 1 -- start start 2024-04-28T12:38:55.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.880+0000 7f2667a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f26601467c0 0x7f2660072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.880+0000 7f2667a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26600b4140 0x7f266006dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.880+0000 7f2667a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2660146ba0 con 0x7f26600b3ce0 2024-04-28T12:38:55.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.880+0000 7f2667a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2660073dd0 con 0x7f26601467c0 2024-04-28T12:38:55.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.880+0000 7f2667a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f26600731c0 con 0x7f26600b4140 2024-04-28T12:38:55.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.881+0000 7f2665000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f26601467c0 0x7f2660072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.881+0000 7f2665000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f26601467c0 0x7f2660072b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41562/0 (socket says 172.21.15.131:41562) 2024-04-28T12:38:55.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.881+0000 7f2665000640 1 -- 172.21.15.131:0/1970431615 learned_addr learned my addr 172.21.15.131:0/1970431615 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:55.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.881+0000 7f2665a00640 1 --2- 172.21.15.131:0/1970431615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26600b4140 0x7f266006dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.881+0000 7f2665000640 1 -- 172.21.15.131:0/1970431615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26600b4140 msgr2=0x7f266006dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:55.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.881+0000 7f2665000640 1 --2- 172.21.15.131:0/1970431615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26600b4140 0x7f266006dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.881+0000 7f2665000640 1 -- 172.21.15.131:0/1970431615 >> v1:172.21.15.157:6789/0 conn(0x7f26600b3ce0 legacy=0x7f26600d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:55.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.881+0000 7f2665000640 1 -- 172.21.15.131:0/1970431615 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26600734a0 con 0x7f26601467c0 2024-04-28T12:38:55.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.882+0000 7f2665000640 1 --2- 172.21.15.131:0/1970431615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26601467c0 0x7f2660072b70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2660073f50 tx=0x7f265402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a050177f55914a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:55.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.883+0000 7f265f400640 1 -- 172.21.15.131:0/1970431615 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654037070 con 0x7f26601467c0 2024-04-28T12:38:55.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.883+0000 7f265f400640 1 -- 172.21.15.131:0/1970431615 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2654032440 con 0x7f26601467c0 2024-04-28T12:38:55.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.883+0000 7f265f400640 1 -- 172.21.15.131:0/1970431615 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654032810 con 0x7f26601467c0 2024-04-28T12:38:55.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.883+0000 7f2667a00640 1 -- 172.21.15.131:0/1970431615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26601467c0 msgr2=0x7f2660072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:55.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.883+0000 7f2667a00640 1 --2- 172.21.15.131:0/1970431615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26601467c0 0x7f2660072b70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2660073f50 tx=0x7f265402a200 comp rx=0 tx=0).stop 2024-04-28T12:38:55.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.883+0000 7f2667a00640 1 -- 172.21.15.131:0/1970431615 shutdown_connections 2024-04-28T12:38:55.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.883+0000 7f2667a00640 1 --2- 172.21.15.131:0/1970431615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26600b4140 0x7f266006dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.883+0000 7f2667a00640 1 --2- 172.21.15.131:0/1970431615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26601467c0 0x7f2660072b70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.883+0000 7f2667a00640 1 -- 172.21.15.131:0/1970431615 >> 172.21.15.131:0/1970431615 conn(0x7f26600b2900 msgr2=0x7f26600b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:55.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.883+0000 7f2667a00640 1 -- 172.21.15.131:0/1970431615 shutdown_connections 2024-04-28T12:38:55.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.883+0000 7f2667a00640 1 -- 172.21.15.131:0/1970431615 wait complete. 2024-04-28T12:38:55.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.885+0000 7f2667a00640 1 Processor -- start 2024-04-28T12:38:55.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.885+0000 7f2667a00640 1 -- start start 2024-04-28T12:38:55.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.886+0000 7f2667a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26600b3ce0 0x7f266015c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.886+0000 7f2667a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26601467c0 0x7f266013fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.886+0000 7f2667a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f266013d9a0 0x7f266013de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.886+0000 7f2667a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26600d1230 con 0x7f266013d9a0 2024-04-28T12:38:55.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.886+0000 7f2667a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2660073a60 con 0x7f26601467c0 2024-04-28T12:38:55.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.886+0000 7f2667a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f26600d13b0 con 0x7f26600b3ce0 2024-04-28T12:38:55.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.886+0000 7f2665a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f266013d9a0 0x7f266013de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.886+0000 7f2665a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f266013d9a0 0x7f266013de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41570/0 (socket says 172.21.15.131:41570) 2024-04-28T12:38:55.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.886+0000 7f2665a00640 1 -- 172.21.15.131:0/1630587442 learned_addr learned my addr 172.21.15.131:0/1630587442 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:55.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f265fe00640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26601467c0 0x7f266013fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f2665000640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26600b3ce0 0x7f266015c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f2665a00640 1 -- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26600b3ce0 msgr2=0x7f266015c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:55.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f2665a00640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26600b3ce0 0x7f266015c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f2665a00640 1 -- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26601467c0 msgr2=0x7f266013fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:55.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f2665a00640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26601467c0 0x7f266013fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:55.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f2665a00640 1 -- 172.21.15.131:0/1630587442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266013c940 con 0x7f266013d9a0 2024-04-28T12:38:55.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f265fe00640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26601467c0 0x7f266013fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:55.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f2665a00640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f266013d9a0 0x7f266013de50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f265800cd60 tx=0x7f265800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:55.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f265cc00640 1 -- 172.21.15.131:0/1630587442 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658003b90 con 0x7f266013d9a0 2024-04-28T12:38:55.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f265cc00640 1 -- 172.21.15.131:0/1630587442 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2658003d50 con 0x7f266013d9a0 2024-04-28T12:38:55.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f265cc00640 1 -- 172.21.15.131:0/1630587442 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658004260 con 0x7f266013d9a0 2024-04-28T12:38:55.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f2667a00640 1 -- 172.21.15.131:0/1630587442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266013cbe0 con 0x7f266013d9a0 2024-04-28T12:38:55.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.887+0000 7f2667a00640 1 -- 172.21.15.131:0/1630587442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f266013d1a0 con 0x7f266013d9a0 2024-04-28T12:38:55.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.889+0000 7f2667a00640 1 -- 172.21.15.131:0/1630587442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2660069350 con 0x7f266013d9a0 2024-04-28T12:38:55.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.890+0000 7f265cc00640 1 -- 172.21.15.131:0/1630587442 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2658004400 con 0x7f266013d9a0 2024-04-28T12:38:55.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.890+0000 7f265cc00640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f264006c920 0x7f264006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:55.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.891+0000 7f265cc00640 1 -- 172.21.15.131:0/1630587442 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f265808ca40 con 0x7f266013d9a0 2024-04-28T12:38:55.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.891+0000 7f2665000640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f264006c920 0x7f264006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:55.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.892+0000 7f2665000640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f264006c920 0x7f264006ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2654034000 tx=0x7f265402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:55.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:55.895+0000 7f265cc00640 1 -- 172.21.15.131:0/1630587442 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f265805aa60 con 0x7f266013d9a0 2024-04-28T12:38:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.094+0000 7f2667a00640 1 -- 172.21.15.131:0/1630587442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26600b2290 con 0x7f266013d9a0 2024-04-28T12:38:56.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.096+0000 7f265cc00640 1 -- 172.21.15.131:0/1630587442 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7f265805a360 con 0x7f266013d9a0 2024-04-28T12:38:56.093 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:56.093 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:56.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.098+0000 7f2667a00640 1 -- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f264006c920 msgr2=0x7f264006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:56.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.098+0000 7f2667a00640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f264006c920 0x7f264006ede0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2654034000 tx=0x7f265402e000 comp rx=0 tx=0).stop 2024-04-28T12:38:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.099+0000 7f2667a00640 1 -- 172.21.15.131:0/1630587442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f266013d9a0 msgr2=0x7f266013de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.099+0000 7f2667a00640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f266013d9a0 0x7f266013de50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f265800cd60 tx=0x7f265800a660 comp rx=0 tx=0).stop 2024-04-28T12:38:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.099+0000 7f2667a00640 1 -- 172.21.15.131:0/1630587442 shutdown_connections 2024-04-28T12:38:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.099+0000 7f2667a00640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f264006c920 0x7f264006ede0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.099+0000 7f2667a00640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f266013d9a0 0x7f266013de50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.099+0000 7f2667a00640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26601467c0 0x7f266013fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.099+0000 7f2667a00640 1 --2- 172.21.15.131:0/1630587442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26600b3ce0 0x7f266015c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.099+0000 7f2667a00640 1 -- 172.21.15.131:0/1630587442 >> 172.21.15.131:0/1630587442 conn(0x7f26600b2900 msgr2=0x7f26600b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.099+0000 7f2667a00640 1 -- 172.21.15.131:0/1630587442 shutdown_connections 2024-04-28T12:38:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.099+0000 7f2667a00640 1 -- 172.21.15.131:0/1630587442 wait complete. 2024-04-28T12:38:56.114 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-28T12:38:56.115 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:38:56.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.321+0000 7fb9a7e00640 1 Processor -- start 2024-04-28T12:38:56.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.321+0000 7fb9a7e00640 1 -- start start 2024-04-28T12:38:56.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.322+0000 7fb9a7e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fb9a815bb90 0x7fb9a80737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:56.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.322+0000 7fb9a7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb9a80af710 0x7fb9a80afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:56.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.323+0000 7fb9a7e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a8072db0 con 0x7fb9a815f610 2024-04-28T12:38:56.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.323+0000 7fb9a7e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a80cec10 con 0x7fb9a80af710 2024-04-28T12:38:56.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.323+0000 7fb9a7e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a80b00b0 con 0x7fb9a815bb90 2024-04-28T12:38:56.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.323+0000 7fb9a6000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb9a80af710 0x7fb9a80afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:56.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.323+0000 7fb9a6000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb9a80af710 0x7fb9a80afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41580/0 (socket says 172.21.15.131:41580) 2024-04-28T12:38:56.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.323+0000 7fb9a6000640 1 -- 172.21.15.131:0/2437679417 learned_addr learned my addr 172.21.15.131:0/2437679417 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:56.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.323+0000 7fb9a6a00640 1 --2- 172.21.15.131:0/2437679417 >> v2:172.21.15.157:3300/0 conn(0x7fb9a815bb90 0x7fb9a80737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:56.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.323+0000 7fb9a6000640 1 -- 172.21.15.131:0/2437679417 >> v2:172.21.15.157:3300/0 conn(0x7fb9a815bb90 msgr2=0x7fb9a80737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:56.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.323+0000 7fb9a6000640 1 --2- 172.21.15.131:0/2437679417 >> v2:172.21.15.157:3300/0 conn(0x7fb9a815bb90 0x7fb9a80737a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.323+0000 7fb9a6000640 1 -- 172.21.15.131:0/2437679417 >> v1:172.21.15.157:6789/0 conn(0x7fb9a815f610 legacy=0x7fb9a815faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:56.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.323+0000 7fb9a6000640 1 -- 172.21.15.131:0/2437679417 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a80cee40 con 0x7fb9a80af710 2024-04-28T12:38:56.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.325+0000 7fb9a6000640 1 --2- 172.21.15.131:0/2437679417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9a80af710 0x7fb9a80afb70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb9a806f820 tx=0x7fb99002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c6a73408ff4b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:56.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.325+0000 7fb9a5600640 1 -- 172.21.15.131:0/2437679417 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990037070 con 0x7fb9a80af710 2024-04-28T12:38:56.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.325+0000 7fb9a5600640 1 -- 172.21.15.131:0/2437679417 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb990032440 con 0x7fb9a80af710 2024-04-28T12:38:56.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.325+0000 7fb9a5600640 1 -- 172.21.15.131:0/2437679417 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990032810 con 0x7fb9a80af710 2024-04-28T12:38:56.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.325+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2437679417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9a80af710 msgr2=0x7fb9a80afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:56.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.325+0000 7fb9a7e00640 1 --2- 172.21.15.131:0/2437679417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9a80af710 0x7fb9a80afb70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb9a806f820 tx=0x7fb99002a230 comp rx=0 tx=0).stop 2024-04-28T12:38:56.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.326+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2437679417 shutdown_connections 2024-04-28T12:38:56.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.326+0000 7fb9a7e00640 1 --2- 172.21.15.131:0/2437679417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9a80af710 0x7fb9a80afb70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.326+0000 7fb9a7e00640 1 --2- 172.21.15.131:0/2437679417 >> v2:172.21.15.157:3300/0 conn(0x7fb9a815bb90 0x7fb9a80737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.326+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2437679417 >> 172.21.15.131:0/2437679417 conn(0x7fb9a8068090 msgr2=0x7fb9a8068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:56.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.326+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2437679417 shutdown_connections 2024-04-28T12:38:56.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.326+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2437679417 wait complete. 2024-04-28T12:38:56.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.327+0000 7fb9a7e00640 1 Processor -- start 2024-04-28T12:38:56.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.327+0000 7fb9a7e00640 1 -- start start 2024-04-28T12:38:56.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.328+0000 7fb9a7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9a80af710 0x7fb9a81f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:56.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.328+0000 7fb9a7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9a815bb90 0x7fb9a815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:56.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.328+0000 7fb9a7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9a815f610 0x7fb9a80b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:56.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.328+0000 7fb9a7e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80d0610 con 0x7fb9a815f610 2024-04-28T12:38:56.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.328+0000 7fb9a7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80725a0 con 0x7fb9a815bb90 2024-04-28T12:38:56.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.328+0000 7fb9a7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a8072ad0 con 0x7fb9a80af710 2024-04-28T12:38:56.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.329+0000 7fb9a7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9a815f610 0x7fb9a80b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:56.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.329+0000 7fb9a7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9a815f610 0x7fb9a80b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41588/0 (socket says 172.21.15.131:41588) 2024-04-28T12:38:56.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.329+0000 7fb9a7400640 1 -- 172.21.15.131:0/2150164788 learned_addr learned my addr 172.21.15.131:0/2150164788 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:56.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.329+0000 7fb9a6000640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9a815bb90 0x7fb9a815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:56.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.329+0000 7fb9a7400640 1 -- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9a80af710 msgr2=0x7fb9a81f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:56.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.329+0000 7fb9a7400640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9a80af710 0x7fb9a81f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.329+0000 7fb9a7400640 1 -- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9a815bb90 msgr2=0x7fb9a815ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:56.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.329+0000 7fb9a7400640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9a815bb90 0x7fb9a815ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.329+0000 7fb9a7400640 1 -- 172.21.15.131:0/2150164788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a8156180 con 0x7fb9a815f610 2024-04-28T12:38:56.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.329+0000 7fb9a7400640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9a815f610 0x7fb9a80b19c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000b970 tx=0x7fb9a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:56.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.330+0000 7fb99ea00640 1 -- 172.21.15.131:0/2150164788 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0019070 con 0x7fb9a815f610 2024-04-28T12:38:56.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.330+0000 7fb99ea00640 1 -- 172.21.15.131:0/2150164788 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a000cb40 con 0x7fb9a815f610 2024-04-28T12:38:56.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.330+0000 7fb99ea00640 1 -- 172.21.15.131:0/2150164788 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0014440 con 0x7fb9a815f610 2024-04-28T12:38:56.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.330+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2150164788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a8156460 con 0x7fb9a815f610 2024-04-28T12:38:56.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.330+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2150164788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a8156a20 con 0x7fb9a815f610 2024-04-28T12:38:56.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.331+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2150164788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a80689a0 con 0x7fb9a815f610 2024-04-28T12:38:56.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.332+0000 7fb99ea00640 1 -- 172.21.15.131:0/2150164788 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9a001c070 con 0x7fb9a815f610 2024-04-28T12:38:56.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.333+0000 7fb99ea00640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb98406c8f0 0x7fb98406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:56.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.333+0000 7fb99ea00640 1 -- 172.21.15.131:0/2150164788 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb9a008bc60 con 0x7fb9a815f610 2024-04-28T12:38:56.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.334+0000 7fb9a6a00640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb98406c8f0 0x7fb98406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:56.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.335+0000 7fb9a6a00640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb98406c8f0 0x7fb98406edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb9980098f0 tx=0x7fb998008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:56.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.337+0000 7fb99ea00640 1 -- 172.21.15.131:0/2150164788 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb9a80689a0 con 0x7fb9a815f610 2024-04-28T12:38:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.560+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2150164788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9a80b1f00 con 0x7fb9a815f610 2024-04-28T12:38:56.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.562+0000 7fb99ea00640 1 -- 172.21.15.131:0/2150164788 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5191 (secure 0 0 0) 0x7fb9a0059bf0 con 0x7fb9a815f610 2024-04-28T12:38:56.558 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:56.558 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1064054651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1064054651},{"type":"v1","addr":"172.21.15.131:6834","nonce":1064054651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/398655652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":398655652},{"type":"v1","addr":"172.21.15.131:6835","nonce":398655652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2619661531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2619661531},{"type":"v1","addr":"172.21.15.157:6837","nonce":2619661531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:37:22.236351+0000","modified":"2024-04-28T12:37:50.533095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1937780695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1937780695},{"type":"v1","addr":"172.21.15.157:6835","nonce":1937780695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34160,"qdb_cluster":[34160]},"id":1}]} 2024-04-28T12:38:56.559 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-04-28T12:38:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.564+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb98406c8f0 msgr2=0x7fb98406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.564+0000 7fb9a7e00640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb98406c8f0 0x7fb98406edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb9980098f0 tx=0x7fb998008040 comp rx=0 tx=0).stop 2024-04-28T12:38:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.564+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2150164788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9a815f610 msgr2=0x7fb9a80b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.564+0000 7fb9a7e00640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9a815f610 0x7fb9a80b19c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000b970 tx=0x7fb9a000be30 comp rx=0 tx=0).stop 2024-04-28T12:38:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.564+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2150164788 shutdown_connections 2024-04-28T12:38:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.564+0000 7fb9a7e00640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb98406c8f0 0x7fb98406edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.564+0000 7fb9a7e00640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9a815f610 0x7fb9a80b19c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.564+0000 7fb9a7e00640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9a815bb90 0x7fb9a815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.564+0000 7fb9a7e00640 1 --2- 172.21.15.131:0/2150164788 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9a80af710 0x7fb9a81f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.564+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2150164788 >> 172.21.15.131:0/2150164788 conn(0x7fb9a8068090 msgr2=0x7fb9a81515e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.564+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2150164788 shutdown_connections 2024-04-28T12:38:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.565+0000 7fb9a7e00640 1 -- 172.21.15.131:0/2150164788 wait complete. 2024-04-28T12:38:56.579 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:38:56.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.783+0000 7f0319600640 1 Processor -- start 2024-04-28T12:38:56.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.783+0000 7f0319600640 1 -- start start 2024-04-28T12:38:56.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.785+0000 7f0319600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0314072b70 0x7f03140d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:56.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.785+0000 7f0319600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f03140d1060 0x7f031406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:56.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.785+0000 7f0319600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0314146ba0 con 0x7f03141467c0 2024-04-28T12:38:56.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.785+0000 7f0319600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0314073dd0 con 0x7f0314072b70 2024-04-28T12:38:56.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.785+0000 7f0319600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f03140731c0 con 0x7f03140d1060 2024-04-28T12:38:56.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.785+0000 7f0312000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0314072b70 0x7f03140d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:56.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.785+0000 7f0312000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0314072b70 0x7f03140d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41590/0 (socket says 172.21.15.131:41590) 2024-04-28T12:38:56.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.785+0000 7f0312000640 1 -- 172.21.15.131:0/4072539760 learned_addr learned my addr 172.21.15.131:0/4072539760 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.785+0000 7f0313400640 1 --2- 172.21.15.131:0/4072539760 >> v2:172.21.15.157:3300/0 conn(0x7f03140d1060 0x7f031406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.785+0000 7f0312000640 1 -- 172.21.15.131:0/4072539760 >> v2:172.21.15.157:3300/0 conn(0x7f03140d1060 msgr2=0x7f031406dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.786+0000 7f0312000640 1 --2- 172.21.15.131:0/4072539760 >> v2:172.21.15.157:3300/0 conn(0x7f03140d1060 0x7f031406dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.786+0000 7f0312000640 1 -- 172.21.15.131:0/4072539760 >> v1:172.21.15.157:6789/0 conn(0x7f03141467c0 legacy=0x7f03140b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.786+0000 7f0312000640 1 -- 172.21.15.131:0/4072539760 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03140734a0 con 0x7f0314072b70 2024-04-28T12:38:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.786+0000 7f0313400640 1 --2- 172.21.15.131:0/4072539760 >> v2:172.21.15.157:3300/0 conn(0x7f03140d1060 0x7f031406dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.787+0000 7f0312000640 1 --2- 172.21.15.131:0/4072539760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0314072b70 0x7f03140d0b20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f03140720f0 tx=0x7f030002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf068617ae842e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:56.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.787+0000 7f0311600640 1 -- 172.21.15.131:0/4072539760 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300037070 con 0x7f0314072b70 2024-04-28T12:38:56.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.787+0000 7f0311600640 1 -- 172.21.15.131:0/4072539760 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0300032440 con 0x7f0314072b70 2024-04-28T12:38:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.787+0000 7f0311600640 1 -- 172.21.15.131:0/4072539760 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300032810 con 0x7f0314072b70 2024-04-28T12:38:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.788+0000 7f0319600640 1 -- 172.21.15.131:0/4072539760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0314072b70 msgr2=0x7f03140d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.788+0000 7f0319600640 1 --2- 172.21.15.131:0/4072539760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0314072b70 0x7f03140d0b20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f03140720f0 tx=0x7f030002a230 comp rx=0 tx=0).stop 2024-04-28T12:38:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.788+0000 7f0319600640 1 -- 172.21.15.131:0/4072539760 shutdown_connections 2024-04-28T12:38:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.788+0000 7f0319600640 1 --2- 172.21.15.131:0/4072539760 >> v2:172.21.15.157:3300/0 conn(0x7f03140d1060 0x7f031406dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.788+0000 7f0319600640 1 --2- 172.21.15.131:0/4072539760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0314072b70 0x7f03140d0b20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.788+0000 7f0319600640 1 -- 172.21.15.131:0/4072539760 >> 172.21.15.131:0/4072539760 conn(0x7f03140b2900 msgr2=0x7f03140b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.788+0000 7f0319600640 1 -- 172.21.15.131:0/4072539760 shutdown_connections 2024-04-28T12:38:56.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.788+0000 7f0319600640 1 -- 172.21.15.131:0/4072539760 wait complete. 2024-04-28T12:38:56.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.790+0000 7f0319600640 1 Processor -- start 2024-04-28T12:38:56.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.790+0000 7f0319600640 1 -- start start 2024-04-28T12:38:56.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0319600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0314072b70 0x7f03141f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0319600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03140d1060 0x7f031415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0319600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03141467c0 0x7f03141e6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0319600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f031406e530 con 0x7f03141467c0 2024-04-28T12:38:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0319600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0314073a60 con 0x7f03140d1060 2024-04-28T12:38:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0319600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f031406e6b0 con 0x7f0314072b70 2024-04-28T12:38:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0313400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03141467c0 0x7f03141e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0313400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03141467c0 0x7f03141e6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41596/0 (socket says 172.21.15.131:41596) 2024-04-28T12:38:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0313400640 1 -- 172.21.15.131:0/1040374236 learned_addr learned my addr 172.21.15.131:0/1040374236 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0312000640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03140d1060 0x7f031415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0313400640 1 -- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0314072b70 msgr2=0x7f03141f6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0312a00640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0314072b70 0x7f03141f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:56.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0313400640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0314072b70 0x7f03141f6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0313400640 1 -- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03140d1060 msgr2=0x7f031415cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0313400640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03140d1060 0x7f031415cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0313400640 1 -- 172.21.15.131:0/1040374236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03141e6750 con 0x7f03141467c0 2024-04-28T12:38:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0312000640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03140d1060 0x7f031415cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.791+0000 7f0312a00640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0314072b70 0x7f03141f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.792+0000 7f0313400640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03141467c0 0x7f03141e6210 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f030c00cef0 tx=0x7f030c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.792+0000 7f030aa00640 1 -- 172.21.15.131:0/1040374236 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c01a070 con 0x7f03141467c0 2024-04-28T12:38:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.792+0000 7f030aa00640 1 -- 172.21.15.131:0/1040374236 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f030c00d860 con 0x7f03141467c0 2024-04-28T12:38:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.792+0000 7f030aa00640 1 -- 172.21.15.131:0/1040374236 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c00dc50 con 0x7f03141467c0 2024-04-28T12:38:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.792+0000 7f0319600640 1 -- 172.21.15.131:0/1040374236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0314142690 con 0x7f03141467c0 2024-04-28T12:38:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.792+0000 7f0319600640 1 -- 172.21.15.131:0/1040374236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0314142c50 con 0x7f03141467c0 2024-04-28T12:38:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.793+0000 7f0319600640 1 -- 172.21.15.131:0/1040374236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0314069350 con 0x7f03141467c0 2024-04-28T12:38:56.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.794+0000 7f030aa00640 1 -- 172.21.15.131:0/1040374236 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f030c01d070 con 0x7f03141467c0 2024-04-28T12:38:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.795+0000 7f030aa00640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f02f406c8f0 0x7f02f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.795+0000 7f030aa00640 1 -- 172.21.15.131:0/1040374236 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f030c08c870 con 0x7f03141467c0 2024-04-28T12:38:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.796+0000 7f0312a00640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f02f406c8f0 0x7f02f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.797+0000 7f0312a00640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f02f406c8f0 0x7f02f406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f030400a810 tx=0x7f0304008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:56.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:56.800+0000 7f030aa00640 1 -- 172.21.15.131:0/1040374236 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f030c05a800 con 0x7f03141467c0 2024-04-28T12:38:57.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.041+0000 7f0319600640 1 -- 172.21.15.131:0/1040374236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03140b2270 con 0x7f03141467c0 2024-04-28T12:38:57.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.043+0000 7f030aa00640 1 -- 172.21.15.131:0/1040374236 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5191 (secure 0 0 0) 0x7f030c05a100 con 0x7f03141467c0 2024-04-28T12:38:57.039 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:57.039 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1064054651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1064054651},{"type":"v1","addr":"172.21.15.131:6834","nonce":1064054651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/398655652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":398655652},{"type":"v1","addr":"172.21.15.131:6835","nonce":398655652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2619661531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6836","nonce":2619661531},{"type":"v1","addr":"172.21.15.157:6837","nonce":2619661531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:37:22.236351+0000","modified":"2024-04-28T12:37:50.533095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/1937780695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1937780695},{"type":"v1","addr":"172.21.15.157:6835","nonce":1937780695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34160,"qdb_cluster":[34160]},"id":1}]} 2024-04-28T12:38:57.040 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-04-28T12:38:57.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.045+0000 7f0319600640 1 -- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f02f406c8f0 msgr2=0x7f02f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.045+0000 7f0319600640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f02f406c8f0 0x7f02f406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f030400a810 tx=0x7f0304008040 comp rx=0 tx=0).stop 2024-04-28T12:38:57.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.045+0000 7f0319600640 1 -- 172.21.15.131:0/1040374236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03141467c0 msgr2=0x7f03141e6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.045+0000 7f0319600640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03141467c0 0x7f03141e6210 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f030c00cef0 tx=0x7f030c00a710 comp rx=0 tx=0).stop 2024-04-28T12:38:57.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.045+0000 7f0319600640 1 -- 172.21.15.131:0/1040374236 shutdown_connections 2024-04-28T12:38:57.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.045+0000 7f0319600640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f02f406c8f0 0x7f02f406edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.045+0000 7f0319600640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03141467c0 0x7f03141e6210 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.045+0000 7f0319600640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03140d1060 0x7f031415cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.045+0000 7f0319600640 1 --2- 172.21.15.131:0/1040374236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0314072b70 0x7f03141f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.045+0000 7f0319600640 1 -- 172.21.15.131:0/1040374236 >> 172.21.15.131:0/1040374236 conn(0x7f03140b2900 msgr2=0x7f03140b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.046+0000 7f0319600640 1 -- 172.21.15.131:0/1040374236 shutdown_connections 2024-04-28T12:38:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.046+0000 7f0319600640 1 -- 172.21.15.131:0/1040374236 wait complete. 2024-04-28T12:38:57.061 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:38:57.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.264+0000 7f5a97400640 1 Processor -- start 2024-04-28T12:38:57.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.264+0000 7f5a97400640 1 -- start start 2024-04-28T12:38:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.266+0000 7f5a97400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5a9014ceb0 0x7f5a900b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.266+0000 7f5a97400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5a900d1570 0x7f5a900cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.266+0000 7f5a97400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a9014d290 con 0x7f5a900cfc90 2024-04-28T12:38:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.266+0000 7f5a97400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5a900d00c0 con 0x7f5a9014ceb0 2024-04-28T12:38:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.266+0000 7f5a97400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5a900b5140 con 0x7f5a900d1570 2024-04-28T12:38:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.266+0000 7f5a94a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5a9014ceb0 0x7f5a900b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.266+0000 7f5a94a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5a9014ceb0 0x7f5a900b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41598/0 (socket says 172.21.15.131:41598) 2024-04-28T12:38:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.266+0000 7f5a94a00640 1 -- 172.21.15.131:0/87158399 learned_addr learned my addr 172.21.15.131:0/87158399 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.267+0000 7f5a87e00640 1 --2- 172.21.15.131:0/87158399 >> v2:172.21.15.157:3300/0 conn(0x7f5a900d1570 0x7f5a900cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.267+0000 7f5a94a00640 1 -- 172.21.15.131:0/87158399 >> v2:172.21.15.157:3300/0 conn(0x7f5a900d1570 msgr2=0x7f5a900cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.267+0000 7f5a94a00640 1 --2- 172.21.15.131:0/87158399 >> v2:172.21.15.157:3300/0 conn(0x7f5a900d1570 0x7f5a900cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.267+0000 7f5a94a00640 1 -- 172.21.15.131:0/87158399 >> v1:172.21.15.131:6789/0 conn(0x7f5a900cfc90 legacy=0x7f5a9006dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.267+0000 7f5a94a00640 1 -- 172.21.15.131:0/87158399 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a900b5420 con 0x7f5a9014ceb0 2024-04-28T12:38:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.267+0000 7f5a87e00640 1 --2- 172.21.15.131:0/87158399 >> v2:172.21.15.157:3300/0 conn(0x7f5a900d1570 0x7f5a900cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.268+0000 7f5a94a00640 1 --2- 172.21.15.131:0/87158399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a9014ceb0 0x7f5a900b4a60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5a9006d1b0 tx=0x7f5a8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a1928450b0ec39 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.268+0000 7f5a8fe00640 1 -- 172.21.15.131:0/87158399 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80037070 con 0x7f5a9014ceb0 2024-04-28T12:38:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.268+0000 7f5a8fe00640 1 -- 172.21.15.131:0/87158399 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a80032440 con 0x7f5a9014ceb0 2024-04-28T12:38:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.268+0000 7f5a8fe00640 1 -- 172.21.15.131:0/87158399 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80032810 con 0x7f5a9014ceb0 2024-04-28T12:38:57.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.269+0000 7f5a97400640 1 -- 172.21.15.131:0/87158399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a9014ceb0 msgr2=0x7f5a900b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.269+0000 7f5a97400640 1 --2- 172.21.15.131:0/87158399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a9014ceb0 0x7f5a900b4a60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5a9006d1b0 tx=0x7f5a8002a270 comp rx=0 tx=0).stop 2024-04-28T12:38:57.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.269+0000 7f5a97400640 1 -- 172.21.15.131:0/87158399 shutdown_connections 2024-04-28T12:38:57.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.269+0000 7f5a97400640 1 --2- 172.21.15.131:0/87158399 >> v2:172.21.15.157:3300/0 conn(0x7f5a900d1570 0x7f5a900cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.269+0000 7f5a97400640 1 --2- 172.21.15.131:0/87158399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a9014ceb0 0x7f5a900b4a60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.269+0000 7f5a97400640 1 -- 172.21.15.131:0/87158399 >> 172.21.15.131:0/87158399 conn(0x7f5a900b2900 msgr2=0x7f5a900b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:57.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.269+0000 7f5a97400640 1 -- 172.21.15.131:0/87158399 shutdown_connections 2024-04-28T12:38:57.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.269+0000 7f5a97400640 1 -- 172.21.15.131:0/87158399 wait complete. 2024-04-28T12:38:57.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.271+0000 7f5a97400640 1 Processor -- start 2024-04-28T12:38:57.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.272+0000 7f5a97400640 1 -- start start 2024-04-28T12:38:57.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.273+0000 7f5a97400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a900cfc90 0x7f5a90148ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.273+0000 7f5a97400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a900d1570 0x7f5a900ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.273+0000 7f5a97400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a9014ceb0 0x7f5a900aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.273+0000 7f5a97400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5a900b5cc0 con 0x7f5a900cfc90 2024-04-28T12:38:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.273+0000 7f5a97400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5a900b59e0 con 0x7f5a9014ceb0 2024-04-28T12:38:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.273+0000 7f5a97400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5a900d1a00 con 0x7f5a900d1570 2024-04-28T12:38:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.273+0000 7f5a94a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a900cfc90 0x7f5a90148ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.273+0000 7f5a94a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a900cfc90 0x7f5a90148ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41602/0 (socket says 172.21.15.131:41602) 2024-04-28T12:38:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.273+0000 7f5a94a00640 1 -- 172.21.15.131:0/2697012160 learned_addr learned my addr 172.21.15.131:0/2697012160 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.274+0000 7f5a87e00640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a900d1570 0x7f5a900ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.274+0000 7f5a94a00640 1 -- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a900d1570 msgr2=0x7f5a900ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.274+0000 7f5a94a00640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a900d1570 0x7f5a900ab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.274+0000 7f5a94a00640 1 -- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a9014ceb0 msgr2=0x7f5a900aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.274+0000 7f5a94a00640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a9014ceb0 0x7f5a900aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.274+0000 7f5a94a00640 1 -- 172.21.15.131:0/2697012160 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a901f9220 con 0x7f5a900cfc90 2024-04-28T12:38:57.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.274+0000 7f5a94a00640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a900cfc90 0x7f5a90148ba0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5a8002a2b0 tx=0x7f5a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.274+0000 7f5a8d600640 1 -- 172.21.15.131:0/2697012160 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80037040 con 0x7f5a900cfc90 2024-04-28T12:38:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.274+0000 7f5a8d600640 1 -- 172.21.15.131:0/2697012160 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a80042070 con 0x7f5a900cfc90 2024-04-28T12:38:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.274+0000 7f5a97400640 1 -- 172.21.15.131:0/2697012160 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a900ac050 con 0x7f5a900cfc90 2024-04-28T12:38:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.274+0000 7f5a8d600640 1 -- 172.21.15.131:0/2697012160 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80030bb0 con 0x7f5a900cfc90 2024-04-28T12:38:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.275+0000 7f5a97400640 1 -- 172.21.15.131:0/2697012160 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a900ac610 con 0x7f5a900cfc90 2024-04-28T12:38:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.276+0000 7f5a97400640 1 -- 172.21.15.131:0/2697012160 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a90076260 con 0x7f5a900cfc90 2024-04-28T12:38:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.277+0000 7f5a8d600640 1 -- 172.21.15.131:0/2697012160 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a80047460 con 0x7f5a900cfc90 2024-04-28T12:38:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.277+0000 7f5a8d600640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5a6c06c940 0x7f5a6c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.278+0000 7f5a8d600640 1 -- 172.21.15.131:0/2697012160 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f5a800acbe0 con 0x7f5a900cfc90 2024-04-28T12:38:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.278+0000 7f5a87e00640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5a6c06c940 0x7f5a6c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:57.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.279+0000 7f5a87e00640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5a6c06c940 0x7f5a6c06ee00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a900b26e0 tx=0x7f5a78008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:57.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.282+0000 7f5a8d600640 1 -- 172.21.15.131:0/2697012160 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5a90076260 con 0x7f5a900cfc90 2024-04-28T12:38:57.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.480+0000 7f5a97400640 1 -- 172.21.15.131:0/2697012160 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a90069350 con 0x7f5a900cfc90 2024-04-28T12:38:57.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.483+0000 7f5a8d600640 1 -- 172.21.15.131:0/2697012160 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18926 (secure 0 0 0) 0x7f5a80045360 con 0x7f5a900cfc90 2024-04-28T12:38:57.481 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:38:57.481 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:38:08.172500+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:37:19.982605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:37:21.207222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:37:24.826402+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:38:57.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.486+0000 7f5a97400640 1 -- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5a6c06c940 msgr2=0x7f5a6c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.486+0000 7f5a97400640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5a6c06c940 0x7f5a6c06ee00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a900b26e0 tx=0x7f5a78008040 comp rx=0 tx=0).stop 2024-04-28T12:38:57.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.486+0000 7f5a97400640 1 -- 172.21.15.131:0/2697012160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a900cfc90 msgr2=0x7f5a90148ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.486+0000 7f5a97400640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a900cfc90 0x7f5a90148ba0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5a8002a2b0 tx=0x7f5a800046c0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.486+0000 7f5a97400640 1 -- 172.21.15.131:0/2697012160 shutdown_connections 2024-04-28T12:38:57.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.486+0000 7f5a97400640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5a6c06c940 0x7f5a6c06ee00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.486+0000 7f5a97400640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5a9014ceb0 0x7f5a900aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.486+0000 7f5a97400640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5a900d1570 0x7f5a900ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.486+0000 7f5a97400640 1 --2- 172.21.15.131:0/2697012160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a900cfc90 0x7f5a90148ba0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.486+0000 7f5a97400640 1 -- 172.21.15.131:0/2697012160 >> 172.21.15.131:0/2697012160 conn(0x7f5a900b2900 msgr2=0x7f5a900727c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:57.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.487+0000 7f5a97400640 1 -- 172.21.15.131:0/2697012160 shutdown_connections 2024-04-28T12:38:57.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.487+0000 7f5a97400640 1 -- 172.21.15.131:0/2697012160 wait complete. 2024-04-28T12:38:57.504 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-28T12:38:57.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.706+0000 7f87b1400640 1 Processor -- start 2024-04-28T12:38:57.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.707+0000 7f87b1400640 1 -- start start 2024-04-28T12:38:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.708+0000 7f87b1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac0d0b40 0x7f87ac0d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.708+0000 7f87b1400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87ac0b0650 con 0x7f87ac15f5c0 2024-04-28T12:38:57.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.708+0000 7f87b1400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87ac163a30 con 0x7f87ac15cab0 2024-04-28T12:38:57.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.708+0000 7f87b1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f87ac0d0900 con 0x7f87ac0d0b40 2024-04-28T12:38:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.709+0000 7f87ab200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f87ac15f5c0 0x7f87ac15fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35148/0 (socket says 172.21.15.131:35148) 2024-04-28T12:38:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.709+0000 7f87ab200640 1 -- 172.21.15.131:0/3591703966 learned_addr learned my addr 172.21.15.131:0/3591703966 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.709+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082086767 0 0) 0x7f87ac0b0650 con 0x7f87ac15f5c0 2024-04-28T12:38:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.709+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8790003520 con 0x7f87ac15f5c0 2024-04-28T12:38:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.709+0000 7f87a3e00640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac0d0b40 0x7f87ac0d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.709+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3763554760 0 0) 0x7f8790003520 con 0x7f87ac15f5c0 2024-04-28T12:38:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.709+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac0d0b40 msgr2=0x7f87ac0d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.709+0000 7f87a9e00640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac0d0b40 0x7f87ac0d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.709+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 >> v1:172.21.15.157:6789/0 conn(0x7f87ac15cab0 legacy=0x7f87ac0afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac162c70 con 0x7f87ac15f5c0 2024-04-28T12:38:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f87a40019f0 con 0x7f87ac15f5c0 2024-04-28T12:38:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 >> v1:172.21.15.131:6789/0 conn(0x7f87ac15f5c0 legacy=0x7f87ac15fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87a9e00640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac0d0b40 0x7f87900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87a9e00640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8790003f60 0x7f8790004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87a9e00640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8790004d40 0x7f87900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f879000ba70 con 0x7f8790004d40 2024-04-28T12:38:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f879000bc20 con 0x7f8790003f60 2024-04-28T12:38:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f879000be20 con 0x7f87ac0d0b40 2024-04-28T12:38:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87ab200640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8790004d40 0x7f87900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87a3e00640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8790003f60 0x7f8790004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87aa800640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac0d0b40 0x7f87900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87ab200640 1 -- 172.21.15.131:0/3591703966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac0d0b40 msgr2=0x7f87900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87ab200640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac0d0b40 0x7f87900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87ab200640 1 -- 172.21.15.131:0/3591703966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8790003f60 msgr2=0x7f8790004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87ab200640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8790003f60 0x7f8790004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.710+0000 7f87ab200640 1 -- 172.21.15.131:0/3591703966 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8790003520 con 0x7f8790004d40 2024-04-28T12:38:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.711+0000 7f87aa800640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac0d0b40 0x7f87900039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.711+0000 7f87ab200640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8790004d40 0x7f87900051c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f87a400a860 tx=0x7f8790005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db95ccad2120a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.712+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a400a480 con 0x7f8790004d40 2024-04-28T12:38:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.712+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a402f070 con 0x7f8790004d40 2024-04-28T12:38:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.712+0000 7f87a9e00640 1 -- 172.21.15.131:0/3591703966 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a402bd20 con 0x7f8790004d40 2024-04-28T12:38:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.712+0000 7f87b1400640 1 -- 172.21.15.131:0/3591703966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8790004d40 msgr2=0x7f87900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.712+0000 7f87b1400640 1 --2- 172.21.15.131:0/3591703966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8790004d40 0x7f87900051c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f87a400a860 tx=0x7f8790005830 comp rx=0 tx=0).stop 2024-04-28T12:38:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.712+0000 7f87ab200640 1 -- 172.21.15.131:0/3591703966 reap_dead start 2024-04-28T12:38:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.712+0000 7f87b1400640 1 -- 172.21.15.131:0/3591703966 shutdown_connections 2024-04-28T12:38:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.712+0000 7f87b1400640 1 -- 172.21.15.131:0/3591703966 >> 172.21.15.131:0/3591703966 conn(0x7f87ac0b2900 msgr2=0x7f87ac0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.712+0000 7f87b1400640 1 -- 172.21.15.131:0/3591703966 shutdown_connections 2024-04-28T12:38:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.712+0000 7f87b1400640 1 -- 172.21.15.131:0/3591703966 wait complete. 2024-04-28T12:38:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.714+0000 7f87b1400640 1 Processor -- start 2024-04-28T12:38:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.714+0000 7f87b1400640 1 -- start start 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.715+0000 7f87b1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8790004d40 0x7f87ac14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.715+0000 7f87b1400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac160970 0x7f87ac06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.715+0000 7f87b1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87ac160df0 0x7f87ac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.715+0000 7f87b1400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac163230 con 0x7f8790004d40 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.715+0000 7f87b1400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac1635a0 con 0x7f87ac160df0 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.715+0000 7f87b1400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f87ac15ce90 con 0x7f87ac160970 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.715+0000 7f87aa800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8790004d40 0x7f87ac14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.716+0000 7f87aa800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8790004d40 0x7f87ac14fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41610/0 (socket says 172.21.15.131:41610) 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.716+0000 7f87aa800640 1 -- 172.21.15.131:0/1801268716 learned_addr learned my addr 172.21.15.131:0/1801268716 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.716+0000 7f87aa800640 1 -- 172.21.15.131:0/1801268716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac160970 msgr2=0x7f87ac06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.716+0000 7f87aa800640 1 --2- 172.21.15.131:0/1801268716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac160970 0x7f87ac06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.716+0000 7f87aa800640 1 -- 172.21.15.131:0/1801268716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87ac160df0 msgr2=0x7f87ac06e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.716+0000 7f87aa800640 1 --2- 172.21.15.131:0/1801268716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87ac160df0 0x7f87ac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.716+0000 7f87aa800640 1 -- 172.21.15.131:0/1801268716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac15fbb0 con 0x7f8790004d40 2024-04-28T12:38:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.716+0000 7f87aa800640 1 --2- 172.21.15.131:0/1801268716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8790004d40 0x7f87ac14fba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f879c00d7a0 tx=0x7f879c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.716+0000 7f87a2a00640 1 -- 172.21.15.131:0/1801268716 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c005070 con 0x7f8790004d40 2024-04-28T12:38:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.717+0000 7f87a2a00640 1 -- 172.21.15.131:0/1801268716 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879c005210 con 0x7f8790004d40 2024-04-28T12:38:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.717+0000 7f87a2a00640 1 -- 172.21.15.131:0/1801268716 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c007030 con 0x7f8790004d40 2024-04-28T12:38:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.717+0000 7f87b1400640 1 -- 172.21.15.131:0/1801268716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ac15fe90 con 0x7f8790004d40 2024-04-28T12:38:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.717+0000 7f87b1400640 1 -- 172.21.15.131:0/1801268716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ac160450 con 0x7f8790004d40 2024-04-28T12:38:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.718+0000 7f87b1400640 1 -- 172.21.15.131:0/1801268716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87ac069350 con 0x7f8790004d40 2024-04-28T12:38:57.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.719+0000 7f87a2a00640 1 -- 172.21.15.131:0/1801268716 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f879c01a030 con 0x7f8790004d40 2024-04-28T12:38:57.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.720+0000 7f87a2a00640 1 --2- 172.21.15.131:0/1801268716 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f878806c920 0x7f878806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:57.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.720+0000 7f87a2a00640 1 -- 172.21.15.131:0/1801268716 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f879c08b690 con 0x7f8790004d40 2024-04-28T12:38:57.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.720+0000 7f87a3e00640 1 --2- 172.21.15.131:0/1801268716 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f878806c920 0x7f878806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:57.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.721+0000 7f87a3e00640 1 --2- 172.21.15.131:0/1801268716 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f878806c920 0x7f878806ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f87ac073380 tx=0x7f8794008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:57.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.724+0000 7f87a2a00640 1 -- 172.21.15.131:0/1801268716 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f879c021020 con 0x7f8790004d40 2024-04-28T12:38:57.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.968+0000 7f87b1400640 1 -- 172.21.15.131:0/1801268716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f87ac15d670 con 0x7f8790004d40 2024-04-28T12:38:57.966 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-04-28T12:38:57.970+0000 7f10b6000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-28T12:38:57.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.974+0000 7f87a2a00640 1 -- 172.21.15.131:0/1801268716 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f879c0596b0 con 0x7f8790004d40 2024-04-28T12:38:57.970 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-28T12:38:57.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.977+0000 7f87b1400640 1 -- 172.21.15.131:0/1801268716 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f878806c920 msgr2=0x7f878806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.977+0000 7f87b1400640 1 --2- 172.21.15.131:0/1801268716 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f878806c920 0x7f878806ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f87ac073380 tx=0x7f8794008040 comp rx=0 tx=0).stop 2024-04-28T12:38:57.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.977+0000 7f87b1400640 1 -- 172.21.15.131:0/1801268716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8790004d40 msgr2=0x7f87ac14fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:57.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.977+0000 7f87b1400640 1 --2- 172.21.15.131:0/1801268716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8790004d40 0x7f87ac14fba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f879c00d7a0 tx=0x7f879c00dc60 comp rx=0 tx=0).stop 2024-04-28T12:38:57.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.977+0000 7f87b1400640 1 -- 172.21.15.131:0/1801268716 shutdown_connections 2024-04-28T12:38:57.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.977+0000 7f87b1400640 1 --2- 172.21.15.131:0/1801268716 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f878806c920 0x7f878806ede0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.977+0000 7f87b1400640 1 --2- 172.21.15.131:0/1801268716 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87ac160df0 0x7f87ac06e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.977+0000 7f87b1400640 1 --2- 172.21.15.131:0/1801268716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87ac160970 0x7f87ac06e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.977+0000 7f87b1400640 1 --2- 172.21.15.131:0/1801268716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8790004d40 0x7f87ac14fba0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:57.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.977+0000 7f87b1400640 1 -- 172.21.15.131:0/1801268716 >> 172.21.15.131:0/1801268716 conn(0x7f87ac0b2900 msgr2=0x7f87ac06f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:57.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.978+0000 7f87b1400640 1 -- 172.21.15.131:0/1801268716 shutdown_connections 2024-04-28T12:38:57.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:57.978+0000 7f87b1400640 1 -- 172.21.15.131:0/1801268716 wait complete. 2024-04-28T12:38:57.993 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-28T12:38:58.010 INFO:tasks.ceph.mds.b.smithi157.stderr: -5970> 2024-04-28T12:37:25.992+0000 7f24c0600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-28T12:38:58.087 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-28T12:38:58.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.191+0000 7f2a86400640 1 Processor -- start 2024-04-28T12:38:58.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.191+0000 7f2a86400640 1 -- start start 2024-04-28T12:38:58.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.193+0000 7f2a86400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f2a800b5250 0x7f2a800cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:58.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.193+0000 7f2a86400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8015d5f0 0x7f2a8015da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:58.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.193+0000 7f2a86400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a8006eaf0 con 0x7f2a800cf440 2024-04-28T12:38:58.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.193+0000 7f2a86400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f2a8006f420 con 0x7f2a800b5250 2024-04-28T12:38:58.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.193+0000 7f2a86400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8006edd0 con 0x7f2a8015d5f0 2024-04-28T12:38:58.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.193+0000 7f2a7f400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2a800cf440 0x7f2a800cf820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35158/0 (socket says 172.21.15.131:35158) 2024-04-28T12:38:58.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.193+0000 7f2a7f400640 1 -- 172.21.15.131:0/3940875083 learned_addr learned my addr 172.21.15.131:0/3940875083 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:38:58.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.193+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3417461420 0 0) 0x7f2a8006eaf0 con 0x7f2a800cf440 2024-04-28T12:38:58.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.193+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a64003520 con 0x7f2a800cf440 2024-04-28T12:38:58.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.193+0000 7f2a7fe00640 1 --2- 172.21.15.131:0/3940875083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8015d5f0 0x7f2a8015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:58.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a77e00640 1 --2- 172.21.15.131:0/3940875083 >> v2:172.21.15.157:3300/0 conn(0x7f2a800b5250 0x7f2a800cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:58.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2544887239 0 0) 0x7f2a64003520 con 0x7f2a800cf440 2024-04-28T12:38:58.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8015d5f0 msgr2=0x7f2a8015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:58.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 --2- 172.21.15.131:0/3940875083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8015d5f0 0x7f2a8015da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:58.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 >> v2:172.21.15.157:3300/0 conn(0x7f2a800b5250 msgr2=0x7f2a800cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:58.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 --2- 172.21.15.131:0/3940875083 >> v2:172.21.15.157:3300/0 conn(0x7f2a800b5250 0x7f2a800cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:58.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8006f0b0 con 0x7f2a800cf440 2024-04-28T12:38:58.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f2a70001670 con 0x7f2a800cf440 2024-04-28T12:38:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 >> v1:172.21.15.131:6789/0 conn(0x7f2a800cf440 legacy=0x7f2a800cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 --2- 172.21.15.131:0/3940875083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a640039b0 0x7f2a64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 --2- 172.21.15.131:0/3940875083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a64004800 0x7f2a64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 --2- 172.21.15.131:0/3940875083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8015d5f0 0x7f2a640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8006f420 con 0x7f2a64004800 2024-04-28T12:38:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8006edd0 con 0x7f2a640039b0 2024-04-28T12:38:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6400bb50 con 0x7f2a8015d5f0 2024-04-28T12:38:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.194+0000 7f2a77e00640 1 --2- 172.21.15.131:0/3940875083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a64004800 0x7f2a64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.195+0000 7f2a7f400640 1 --2- 172.21.15.131:0/3940875083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a640039b0 0x7f2a64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:58.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.195+0000 7f2a7fe00640 1 --2- 172.21.15.131:0/3940875083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8015d5f0 0x7f2a640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:58.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.195+0000 7f2a77e00640 1 -- 172.21.15.131:0/3940875083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8015d5f0 msgr2=0x7f2a640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:58.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.195+0000 7f2a77e00640 1 --2- 172.21.15.131:0/3940875083 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a8015d5f0 0x7f2a640051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:58.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.195+0000 7f2a77e00640 1 -- 172.21.15.131:0/3940875083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a640039b0 msgr2=0x7f2a64003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:58.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.195+0000 7f2a77e00640 1 --2- 172.21.15.131:0/3940875083 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a640039b0 0x7f2a64003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:58.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.195+0000 7f2a77e00640 1 -- 172.21.15.131:0/3940875083 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a64003520 con 0x7f2a64004800 2024-04-28T12:38:58.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.196+0000 7f2a77e00640 1 --2- 172.21.15.131:0/3940875083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a64004800 0x7f2a64004c80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800ba00 tx=0x7f2a64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa906fe657071f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:58.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.196+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68030070 con 0x7f2a64004800 2024-04-28T12:38:58.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.196+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6802adb0 con 0x7f2a64004800 2024-04-28T12:38:58.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.196+0000 7f2a7ea00640 1 -- 172.21.15.131:0/3940875083 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800b600 con 0x7f2a64004800 2024-04-28T12:38:58.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.196+0000 7f2a86400640 1 -- 172.21.15.131:0/3940875083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a64004800 msgr2=0x7f2a64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:58.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.196+0000 7f2a86400640 1 --2- 172.21.15.131:0/3940875083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a64004800 0x7f2a64004c80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800ba00 tx=0x7f2a64005830 comp rx=0 tx=0).stop 2024-04-28T12:38:58.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.196+0000 7f2a7fe00640 1 -- 172.21.15.131:0/3940875083 reap_dead start 2024-04-28T12:38:58.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.196+0000 7f2a86400640 1 -- 172.21.15.131:0/3940875083 shutdown_connections 2024-04-28T12:38:58.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.196+0000 7f2a86400640 1 -- 172.21.15.131:0/3940875083 >> 172.21.15.131:0/3940875083 conn(0x7f2a800b2900 msgr2=0x7f2a800b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:58.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.197+0000 7f2a86400640 1 -- 172.21.15.131:0/3940875083 shutdown_connections 2024-04-28T12:38:58.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.197+0000 7f2a86400640 1 -- 172.21.15.131:0/3940875083 wait complete. 2024-04-28T12:38:58.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.198+0000 7f2a86400640 1 Processor -- start 2024-04-28T12:38:58.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.198+0000 7f2a86400640 1 -- start start 2024-04-28T12:38:58.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.199+0000 7f2a86400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a64004800 0x7f2a80154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:58.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.199+0000 7f2a86400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a80152980 0x7f2a80157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:58.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.199+0000 7f2a86400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a80157c70 0x7f2a801554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:58.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.199+0000 7f2a86400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a800d0470 con 0x7f2a80157c70 2024-04-28T12:38:58.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.199+0000 7f2a86400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8006f8b0 con 0x7f2a64004800 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.199+0000 7f2a86400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8015da80 con 0x7f2a80152980 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.199+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a80157c70 0x7f2a801554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.200+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a80157c70 0x7f2a801554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41630/0 (socket says 172.21.15.131:41630) 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.200+0000 7f2a7fe00640 1 -- 172.21.15.131:0/971970928 learned_addr learned my addr 172.21.15.131:0/971970928 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.200+0000 7f2a77e00640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a80152980 0x7f2a80157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.200+0000 7f2a7fe00640 1 -- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a80152980 msgr2=0x7f2a80157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.200+0000 7f2a7fe00640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a80152980 0x7f2a80157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.200+0000 7f2a7fe00640 1 -- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a64004800 msgr2=0x7f2a80154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.200+0000 7f2a7fe00640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a64004800 0x7f2a80154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.200+0000 7f2a7fe00640 1 -- 172.21.15.131:0/971970928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a80152e10 con 0x7f2a80157c70 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.200+0000 7f2a7fe00640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a80157c70 0x7f2a801554f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2a800b0840 tx=0x7f2a7800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.200+0000 7f2a77400640 1 -- 172.21.15.131:0/971970928 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78018020 con 0x7f2a80157c70 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.201+0000 7f2a77400640 1 -- 172.21.15.131:0/971970928 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a7800fe20 con 0x7f2a80157c70 2024-04-28T12:38:58.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.201+0000 7f2a77400640 1 -- 172.21.15.131:0/971970928 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78014630 con 0x7f2a80157c70 2024-04-28T12:38:58.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.201+0000 7f2a86400640 1 -- 172.21.15.131:0/971970928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a801530f0 con 0x7f2a80157c70 2024-04-28T12:38:58.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.201+0000 7f2a86400640 1 -- 172.21.15.131:0/971970928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a801536b0 con 0x7f2a80157c70 2024-04-28T12:38:58.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.202+0000 7f2a86400640 1 -- 172.21.15.131:0/971970928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a8006eaf0 con 0x7f2a80157c70 2024-04-28T12:38:58.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.203+0000 7f2a77400640 1 -- 172.21.15.131:0/971970928 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a7801b020 con 0x7f2a80157c70 2024-04-28T12:38:58.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.203+0000 7f2a77400640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2a5c06c8f0 0x7f2a5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:58.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.204+0000 7f2a77400640 1 -- 172.21.15.131:0/971970928 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f2a7808c370 con 0x7f2a80157c70 2024-04-28T12:38:58.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.204+0000 7f2a7f400640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2a5c06c8f0 0x7f2a5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:58.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.205+0000 7f2a7f400640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2a5c06c8f0 0x7f2a5c06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a70001eb0 tx=0x7f2a70009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:58.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.208+0000 7f2a77400640 1 -- 172.21.15.131:0/971970928 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a8006eaf0 con 0x7f2a80157c70 2024-04-28T12:38:58.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.451+0000 7f2a86400640 1 -- 172.21.15.131:0/971970928 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2a80069350 con 0x7f2a80157c70 2024-04-28T12:38:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.978+0000 7f2a77400640 1 -- 172.21.15.131:0/971970928 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f2a78022030 con 0x7f2a80157c70 2024-04-28T12:38:58.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.981+0000 7f2a86400640 1 -- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2a5c06c8f0 msgr2=0x7f2a5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:58.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.981+0000 7f2a86400640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2a5c06c8f0 0x7f2a5c06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a70001eb0 tx=0x7f2a70009040 comp rx=0 tx=0).stop 2024-04-28T12:38:58.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.981+0000 7f2a86400640 1 -- 172.21.15.131:0/971970928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a80157c70 msgr2=0x7f2a801554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:58.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.981+0000 7f2a86400640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a80157c70 0x7f2a801554f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2a800b0840 tx=0x7f2a7800f520 comp rx=0 tx=0).stop 2024-04-28T12:38:58.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.982+0000 7f2a86400640 1 -- 172.21.15.131:0/971970928 shutdown_connections 2024-04-28T12:38:58.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.982+0000 7f2a86400640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f2a5c06c8f0 0x7f2a5c06edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:58.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.982+0000 7f2a86400640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a80157c70 0x7f2a801554f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:58.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.982+0000 7f2a86400640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a80152980 0x7f2a80157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:58.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.982+0000 7f2a86400640 1 --2- 172.21.15.131:0/971970928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a64004800 0x7f2a80154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:58.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.982+0000 7f2a86400640 1 -- 172.21.15.131:0/971970928 >> 172.21.15.131:0/971970928 conn(0x7f2a800b2900 msgr2=0x7f2a800b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:58.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.982+0000 7f2a86400640 1 -- 172.21.15.131:0/971970928 shutdown_connections 2024-04-28T12:38:58.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:58.982+0000 7f2a86400640 1 -- 172.21.15.131:0/971970928 wait complete. 2024-04-28T12:38:59.000 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-28T12:38:59.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.211+0000 7f483b200640 1 Processor -- start 2024-04-28T12:38:59.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.211+0000 7f483b200640 1 -- start start 2024-04-28T12:38:59.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.212+0000 7f483b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c144a90 0x7f483c0d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:59.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.212+0000 7f483b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f483c0d13d0 0x7f483c0b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:59.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.212+0000 7f483b200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f483c160780 con 0x7f483c0b5010 2024-04-28T12:38:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.212+0000 7f483b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f483c0d1830 con 0x7f483c0d13d0 2024-04-28T12:38:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.212+0000 7f483b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f483c0b57e0 con 0x7f483c144a90 2024-04-28T12:38:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.212+0000 7f4833e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f483c0d13d0 0x7f483c0b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.212+0000 7f4833e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f483c0d13d0 0x7f483c0b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41644/0 (socket says 172.21.15.131:41644) 2024-04-28T12:38:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.212+0000 7f4833e00640 1 -- 172.21.15.131:0/3896968953 learned_addr learned my addr 172.21.15.131:0/3896968953 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.213+0000 7f4839e00640 1 --2- 172.21.15.131:0/3896968953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c144a90 0x7f483c0d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.213+0000 7f4833e00640 1 -- 172.21.15.131:0/3896968953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c144a90 msgr2=0x7f483c0d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:38:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.213+0000 7f4833e00640 1 --2- 172.21.15.131:0/3896968953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c144a90 0x7f483c0d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.213+0000 7f4833e00640 1 -- 172.21.15.131:0/3896968953 >> v1:172.21.15.131:6789/0 conn(0x7f483c0b5010 legacy=0x7f483c06ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.213+0000 7f4833e00640 1 -- 172.21.15.131:0/3896968953 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c0cdf30 con 0x7f483c0d13d0 2024-04-28T12:38:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.213+0000 7f4839e00640 1 --2- 172.21.15.131:0/3896968953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c144a90 0x7f483c0d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:38:59.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.214+0000 7f4833e00640 1 --2- 172.21.15.131:0/3896968953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c0d13d0 0x7f483c0b4ad0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f48240098a0 tx=0x7f482402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e191b883458e1861 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.214+0000 7f4839400640 1 -- 172.21.15.131:0/3896968953 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824037070 con 0x7f483c0d13d0 2024-04-28T12:38:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.214+0000 7f4839400640 1 -- 172.21.15.131:0/3896968953 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4824032440 con 0x7f483c0d13d0 2024-04-28T12:38:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.214+0000 7f4839400640 1 -- 172.21.15.131:0/3896968953 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824032810 con 0x7f483c0d13d0 2024-04-28T12:38:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.215+0000 7f483b200640 1 -- 172.21.15.131:0/3896968953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c0d13d0 msgr2=0x7f483c0b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.215+0000 7f483b200640 1 --2- 172.21.15.131:0/3896968953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c0d13d0 0x7f483c0b4ad0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f48240098a0 tx=0x7f482402a1f0 comp rx=0 tx=0).stop 2024-04-28T12:38:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.215+0000 7f483b200640 1 -- 172.21.15.131:0/3896968953 shutdown_connections 2024-04-28T12:38:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.215+0000 7f483b200640 1 --2- 172.21.15.131:0/3896968953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c0d13d0 0x7f483c0b4ad0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.215+0000 7f483b200640 1 --2- 172.21.15.131:0/3896968953 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c144a90 0x7f483c0d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.215+0000 7f483b200640 1 -- 172.21.15.131:0/3896968953 >> 172.21.15.131:0/3896968953 conn(0x7f483c0684b0 msgr2=0x7f483c0688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.215+0000 7f483b200640 1 -- 172.21.15.131:0/3896968953 shutdown_connections 2024-04-28T12:38:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.215+0000 7f483b200640 1 -- 172.21.15.131:0/3896968953 wait complete. 2024-04-28T12:38:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.217+0000 7f483b200640 1 Processor -- start 2024-04-28T12:38:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.217+0000 7f483b200640 1 -- start start 2024-04-28T12:38:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.218+0000 7f483b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c0b5010 0x7f483c0b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.218+0000 7f483b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c0d13d0 0x7f483c0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.218+0000 7f483b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f483c144a90 0x7f483c1f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.218+0000 7f483b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f483c15ff80 con 0x7f483c0b5010 2024-04-28T12:38:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.218+0000 7f483b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f483c15fca0 con 0x7f483c144a90 2024-04-28T12:38:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.218+0000 7f483b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f483c160260 con 0x7f483c0d13d0 2024-04-28T12:38:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.218+0000 7f4839e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c0b5010 0x7f483c0b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.218+0000 7f4839e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c0b5010 0x7f483c0b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41646/0 (socket says 172.21.15.131:41646) 2024-04-28T12:38:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.218+0000 7f4839e00640 1 -- 172.21.15.131:0/56627096 learned_addr learned my addr 172.21.15.131:0/56627096 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:38:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.219+0000 7f4839e00640 1 -- 172.21.15.131:0/56627096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c0d13d0 msgr2=0x7f483c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.219+0000 7f4839e00640 1 --2- 172.21.15.131:0/56627096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c0d13d0 0x7f483c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.219+0000 7f4839e00640 1 -- 172.21.15.131:0/56627096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f483c144a90 msgr2=0x7f483c1f8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.219+0000 7f4839e00640 1 --2- 172.21.15.131:0/56627096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f483c144a90 0x7f483c1f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.219+0000 7f4839e00640 1 -- 172.21.15.131:0/56627096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c0cfcc0 con 0x7f483c0b5010 2024-04-28T12:38:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.219+0000 7f4839e00640 1 --2- 172.21.15.131:0/56627096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c0b5010 0x7f483c0b0020 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f482c00c820 tx=0x7f482c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.219+0000 7f4832000640 1 -- 172.21.15.131:0/56627096 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c015070 con 0x7f483c0b5010 2024-04-28T12:38:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.219+0000 7f4832000640 1 -- 172.21.15.131:0/56627096 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f482c007a30 con 0x7f483c0b5010 2024-04-28T12:38:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.219+0000 7f4832000640 1 -- 172.21.15.131:0/56627096 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c007de0 con 0x7f483c0b5010 2024-04-28T12:38:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.220+0000 7f483b200640 1 -- 172.21.15.131:0/56627096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483c0cffa0 con 0x7f483c0b5010 2024-04-28T12:38:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.220+0000 7f483b200640 1 -- 172.21.15.131:0/56627096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483c1f85d0 con 0x7f483c0b5010 2024-04-28T12:38:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.221+0000 7f483b200640 1 -- 172.21.15.131:0/56627096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483c069350 con 0x7f483c0b5010 2024-04-28T12:38:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.222+0000 7f4832000640 1 -- 172.21.15.131:0/56627096 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f482c018070 con 0x7f483c0b5010 2024-04-28T12:38:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.223+0000 7f4832000640 1 --2- 172.21.15.131:0/56627096 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f481806c8f0 0x7f481806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:38:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.223+0000 7f4832000640 1 -- 172.21.15.131:0/56627096 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f482c08be30 con 0x7f483c0b5010 2024-04-28T12:38:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.223+0000 7f4833e00640 1 --2- 172.21.15.131:0/56627096 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f481806c8f0 0x7f481806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:38:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.225+0000 7f4833e00640 1 --2- 172.21.15.131:0/56627096 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f481806c8f0 0x7f481806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4824034000 tx=0x7f482402c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:38:59.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.227+0000 7f4832000640 1 -- 172.21.15.131:0/56627096 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f482c059e90 con 0x7f483c0b5010 2024-04-28T12:38:59.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.426+0000 7f483b200640 1 -- 172.21.15.131:0/56627096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f483c072960 con 0x7f483c0b5010 2024-04-28T12:38:59.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.982+0000 7f4832000640 1 -- 172.21.15.131:0/56627096 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7f482c059790 con 0x7f483c0b5010 2024-04-28T12:38:59.979 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-04-28T12:38:59.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.986+0000 7f483b200640 1 -- 172.21.15.131:0/56627096 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f481806c8f0 msgr2=0x7f481806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:59.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.986+0000 7f483b200640 1 --2- 172.21.15.131:0/56627096 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f481806c8f0 0x7f481806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4824034000 tx=0x7f482402c000 comp rx=0 tx=0).stop 2024-04-28T12:38:59.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.986+0000 7f483b200640 1 -- 172.21.15.131:0/56627096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c0b5010 msgr2=0x7f483c0b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:38:59.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.986+0000 7f483b200640 1 --2- 172.21.15.131:0/56627096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c0b5010 0x7f483c0b0020 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f482c00c820 tx=0x7f482c00cce0 comp rx=0 tx=0).stop 2024-04-28T12:38:59.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.987+0000 7f483b200640 1 -- 172.21.15.131:0/56627096 shutdown_connections 2024-04-28T12:38:59.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.987+0000 7f483b200640 1 --2- 172.21.15.131:0/56627096 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f481806c8f0 0x7f481806edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:59.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.987+0000 7f483b200640 1 --2- 172.21.15.131:0/56627096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f483c144a90 0x7f483c1f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:59.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.987+0000 7f483b200640 1 --2- 172.21.15.131:0/56627096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f483c0d13d0 0x7f483c0ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:59.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.987+0000 7f483b200640 1 --2- 172.21.15.131:0/56627096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c0b5010 0x7f483c0b0020 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:38:59.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.987+0000 7f483b200640 1 -- 172.21.15.131:0/56627096 >> 172.21.15.131:0/56627096 conn(0x7f483c0684b0 msgr2=0x7f483c06db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:38:59.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.989+0000 7f483b200640 1 -- 172.21.15.131:0/56627096 shutdown_connections 2024-04-28T12:38:59.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:38:59.989+0000 7f483b200640 1 -- 172.21.15.131:0/56627096 wait complete. 2024-04-28T12:39:00.014 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-28T12:39:00.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.208+0000 7f31a7600640 1 Processor -- start 2024-04-28T12:39:00.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.208+0000 7f31a7600640 1 -- start start 2024-04-28T12:39:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.209+0000 7f31a7600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f31a0146800 0x7f31a00cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:00.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.210+0000 7f31a7600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a006df30 con 0x7f31a0071e60 2024-04-28T12:39:00.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.210+0000 7f31a7600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a006e210 con 0x7f31a006fec0 2024-04-28T12:39:00.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.210+0000 7f31a7600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f31a0146be0 con 0x7f31a0146800 2024-04-28T12:39:00.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.210+0000 7f31a4c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f31a0146800 0x7f31a00cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:00.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.210+0000 7f31a4c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f31a0146800 0x7f31a00cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41650/0 (socket says 172.21.15.131:41650) 2024-04-28T12:39:00.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.210+0000 7f31a4c00640 1 -- 172.21.15.131:0/3431358977 learned_addr learned my addr 172.21.15.131:0/3431358977 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:00.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.210+0000 7f31a5600640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f31a0071e60 0x7f31a0160aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35168/0 (socket says 172.21.15.131:35168) 2024-04-28T12:39:00.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.210+0000 7f31a4c00640 1 -- 172.21.15.131:0/3431358977 >> v1:172.21.15.157:6789/0 conn(0x7f31a006fec0 legacy=0x7f31a006ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:00.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.211+0000 7f31a4c00640 1 -- 172.21.15.131:0/3431358977 >> v1:172.21.15.131:6789/0 conn(0x7f31a0071e60 legacy=0x7f31a0160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:00.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.211+0000 7f31a4c00640 1 -- 172.21.15.131:0/3431358977 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a006e4f0 con 0x7f31a0146800 2024-04-28T12:39:00.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.211+0000 7f319fe00640 1 -- 172.21.15.131:0/3431358977 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1185445753 0 0) 0x7f31a006df30 con 0x7f31a0071e60 2024-04-28T12:39:00.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.212+0000 7f31a4c00640 1 --2- 172.21.15.131:0/3431358977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31a0146800 0x7f31a00cfcd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3190009870 tx=0x7f319002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cb76fe2ab768bc server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:00.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.212+0000 7f319fe00640 1 -- 172.21.15.131:0/3431358977 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190037070 con 0x7f31a0146800 2024-04-28T12:39:00.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.212+0000 7f319fe00640 1 -- 172.21.15.131:0/3431358977 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3190032440 con 0x7f31a0146800 2024-04-28T12:39:00.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.212+0000 7f319fe00640 1 -- 172.21.15.131:0/3431358977 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190032810 con 0x7f31a0146800 2024-04-28T12:39:00.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.212+0000 7f31a7600640 1 -- 172.21.15.131:0/3431358977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31a0146800 msgr2=0x7f31a00cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:00.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.212+0000 7f31a7600640 1 --2- 172.21.15.131:0/3431358977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31a0146800 0x7f31a00cfcd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3190009870 tx=0x7f319002a2a0 comp rx=0 tx=0).stop 2024-04-28T12:39:00.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.213+0000 7f31a7600640 1 -- 172.21.15.131:0/3431358977 shutdown_connections 2024-04-28T12:39:00.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.213+0000 7f31a7600640 1 --2- 172.21.15.131:0/3431358977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31a0146800 0x7f31a00cfcd0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:00.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.213+0000 7f31a7600640 1 -- 172.21.15.131:0/3431358977 >> 172.21.15.131:0/3431358977 conn(0x7f31a00b2790 msgr2=0x7f31a00b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:00.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.213+0000 7f31a7600640 1 -- 172.21.15.131:0/3431358977 shutdown_connections 2024-04-28T12:39:00.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.213+0000 7f31a7600640 1 -- 172.21.15.131:0/3431358977 wait complete. 2024-04-28T12:39:00.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.214+0000 7f31a7600640 1 Processor -- start 2024-04-28T12:39:00.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.214+0000 7f31a7600640 1 -- start start 2024-04-28T12:39:00.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.215+0000 7f31a7600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31a006fec0 0x7f31a01566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:00.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.215+0000 7f31a7600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31a0071e60 0x7f31a0143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:00.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.215+0000 7f31a7600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31a0146800 0x7f31a01432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:00.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.215+0000 7f31a7600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f31a00d0520 con 0x7f31a006fec0 2024-04-28T12:39:00.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.215+0000 7f31a7600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f31a006e860 con 0x7f31a0071e60 2024-04-28T12:39:00.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.215+0000 7f31a7600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f31a006f400 con 0x7f31a0146800 2024-04-28T12:39:00.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.216+0000 7f31a4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31a006fec0 0x7f31a01566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:00.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.216+0000 7f31a4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31a006fec0 0x7f31a01566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41652/0 (socket says 172.21.15.131:41652) 2024-04-28T12:39:00.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.216+0000 7f31a4c00640 1 -- 172.21.15.131:0/774132050 learned_addr learned my addr 172.21.15.131:0/774132050 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:00.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.216+0000 7f31a5600640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31a0146800 0x7f31a01432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:00.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.216+0000 7f3197e00640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31a0071e60 0x7f31a0143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:00.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.216+0000 7f31a4c00640 1 -- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31a0146800 msgr2=0x7f31a01432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:00.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.216+0000 7f31a4c00640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31a0146800 0x7f31a01432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:00.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.216+0000 7f31a4c00640 1 -- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31a0071e60 msgr2=0x7f31a0143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:00.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.216+0000 7f31a4c00640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31a0071e60 0x7f31a0143d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:00.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.216+0000 7f31a4c00640 1 -- 172.21.15.131:0/774132050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a0142200 con 0x7f31a006fec0 2024-04-28T12:39:00.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.217+0000 7f31a4c00640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31a006fec0 0x7f31a01566e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3190009870 tx=0x7f31900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:00.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.217+0000 7f319d600640 1 -- 172.21.15.131:0/774132050 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190041070 con 0x7f31a006fec0 2024-04-28T12:39:00.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.217+0000 7f319d600640 1 -- 172.21.15.131:0/774132050 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f319003c870 con 0x7f31a006fec0 2024-04-28T12:39:00.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.217+0000 7f319d600640 1 -- 172.21.15.131:0/774132050 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190037040 con 0x7f31a006fec0 2024-04-28T12:39:00.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.217+0000 7f31a7600640 1 -- 172.21.15.131:0/774132050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a01424e0 con 0x7f31a006fec0 2024-04-28T12:39:00.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.217+0000 7f31a7600640 1 -- 172.21.15.131:0/774132050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a0142aa0 con 0x7f31a006fec0 2024-04-28T12:39:00.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.218+0000 7f31a7600640 1 -- 172.21.15.131:0/774132050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a0069350 con 0x7f31a006fec0 2024-04-28T12:39:00.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.219+0000 7f319d600640 1 -- 172.21.15.131:0/774132050 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3190044070 con 0x7f31a006fec0 2024-04-28T12:39:00.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.220+0000 7f319d600640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f317c06c920 0x7f317c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:00.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.220+0000 7f319d600640 1 -- 172.21.15.131:0/774132050 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6383+0+0 (secure 0 0 0) 0x7f31900ac9b0 con 0x7f31a006fec0 2024-04-28T12:39:00.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.221+0000 7f3197e00640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f317c06c920 0x7f317c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:00.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.222+0000 7f3197e00640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f317c06c920 0x7f317c06ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f31a0136330 tx=0x7f3188008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:00.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.225+0000 7f319d600640 1 -- 172.21.15.131:0/774132050 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f319007ac60 con 0x7f31a006fec0 2024-04-28T12:39:00.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.435+0000 7f31a7600640 1 -- 172.21.15.131:0/774132050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f31a0073b10 con 0x7f31a006fec0 2024-04-28T12:39:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.985+0000 7f319d600640 1 -- 172.21.15.131:0/774132050 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7f319007a560 con 0x7f31a006fec0 2024-04-28T12:39:00.983 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-04-28T12:39:00.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.989+0000 7f31a7600640 1 -- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f317c06c920 msgr2=0x7f317c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:00.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.989+0000 7f31a7600640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f317c06c920 0x7f317c06ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f31a0136330 tx=0x7f3188008040 comp rx=0 tx=0).stop 2024-04-28T12:39:00.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.989+0000 7f31a7600640 1 -- 172.21.15.131:0/774132050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31a006fec0 msgr2=0x7f31a01566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:00.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.989+0000 7f31a7600640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31a006fec0 0x7f31a01566e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3190009870 tx=0x7f31900046c0 comp rx=0 tx=0).stop 2024-04-28T12:39:00.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.989+0000 7f31a7600640 1 -- 172.21.15.131:0/774132050 shutdown_connections 2024-04-28T12:39:00.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.989+0000 7f31a7600640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f317c06c920 0x7f317c06ede0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:00.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.989+0000 7f31a7600640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f31a0146800 0x7f31a01432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:00.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.989+0000 7f31a7600640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31a0071e60 0x7f31a0143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:00.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.989+0000 7f31a7600640 1 --2- 172.21.15.131:0/774132050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31a006fec0 0x7f31a01566e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:00.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.990+0000 7f31a7600640 1 -- 172.21.15.131:0/774132050 >> 172.21.15.131:0/774132050 conn(0x7f31a00b2790 msgr2=0x7f31a0074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:00.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.990+0000 7f31a7600640 1 -- 172.21.15.131:0/774132050 shutdown_connections 2024-04-28T12:39:00.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:00.990+0000 7f31a7600640 1 -- 172.21.15.131:0/774132050 wait complete. 2024-04-28T12:39:01.011 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-28T12:39:01.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.215+0000 7fca8ea00640 1 Processor -- start 2024-04-28T12:39:01.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.216+0000 7fca8ea00640 1 -- start start 2024-04-28T12:39:01.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.217+0000 7fca8ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fca8815f050 0x7fca8806de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:01.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.217+0000 7fca8ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806e3b0 0x7fca880d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:01.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.217+0000 7fca8ea00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fca8815f430 0x7fca880d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:01.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.217+0000 7fca8ea00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fca880d21b0 con 0x7fca8815f050 2024-04-28T12:39:01.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.217+0000 7fca8ea00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fca880d1110 con 0x7fca8815f430 2024-04-28T12:39:01.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.217+0000 7fca8ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fca880d13f0 con 0x7fca8806e3b0 2024-04-28T12:39:01.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.218+0000 7fca87e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fca8815f050 0x7fca8806de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:01.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.218+0000 7fca87e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fca8815f050 0x7fca8806de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47968/0 (socket says 172.21.15.131:47968) 2024-04-28T12:39:01.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.218+0000 7fca87e00640 1 -- 172.21.15.131:0/2654203255 learned_addr learned my addr 172.21.15.131:0/2654203255 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:01.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.218+0000 7fca87400640 1 --2- 172.21.15.131:0/2654203255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806e3b0 0x7fca880d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.218+0000 7fca87e00640 1 -- 172.21.15.131:0/2654203255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806e3b0 msgr2=0x7fca880d2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.218+0000 7fca87e00640 1 --2- 172.21.15.131:0/2654203255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806e3b0 0x7fca880d2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.218+0000 7fca8ca00640 1 --2- 172.21.15.131:0/2654203255 >> v2:172.21.15.157:3300/0 conn(0x7fca8815f430 0x7fca880d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.218+0000 7fca87e00640 1 -- 172.21.15.131:0/2654203255 >> v2:172.21.15.157:3300/0 conn(0x7fca8815f430 msgr2=0x7fca880d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.218+0000 7fca87e00640 1 --2- 172.21.15.131:0/2654203255 >> v2:172.21.15.157:3300/0 conn(0x7fca8815f430 0x7fca880d31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.218+0000 7fca87e00640 1 -- 172.21.15.131:0/2654203255 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca880d16d0 con 0x7fca8815f050 2024-04-28T12:39:01.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.219+0000 7fca87e00640 1 --2- 172.21.15.131:0/2654203255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8815f050 0x7fca8806de70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fca88072aa0 tx=0x7fca7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418612220c09eda3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:01.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.220+0000 7fca86a00640 1 -- 172.21.15.131:0/2654203255 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78037070 con 0x7fca8815f050 2024-04-28T12:39:01.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.220+0000 7fca86a00640 1 -- 172.21.15.131:0/2654203255 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca78032440 con 0x7fca8815f050 2024-04-28T12:39:01.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.220+0000 7fca86a00640 1 -- 172.21.15.131:0/2654203255 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78032810 con 0x7fca8815f050 2024-04-28T12:39:01.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.220+0000 7fca8ea00640 1 -- 172.21.15.131:0/2654203255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8815f050 msgr2=0x7fca8806de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:01.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.220+0000 7fca8ea00640 1 --2- 172.21.15.131:0/2654203255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8815f050 0x7fca8806de70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fca88072aa0 tx=0x7fca7802a1e0 comp rx=0 tx=0).stop 2024-04-28T12:39:01.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.220+0000 7fca8ea00640 1 -- 172.21.15.131:0/2654203255 shutdown_connections 2024-04-28T12:39:01.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.220+0000 7fca8ea00640 1 --2- 172.21.15.131:0/2654203255 >> v2:172.21.15.157:3300/0 conn(0x7fca8815f430 0x7fca880d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:01.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.220+0000 7fca8ea00640 1 --2- 172.21.15.131:0/2654203255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8806e3b0 0x7fca880d2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:01.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.220+0000 7fca8ea00640 1 --2- 172.21.15.131:0/2654203255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8815f050 0x7fca8806de70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:01.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.220+0000 7fca8ea00640 1 -- 172.21.15.131:0/2654203255 >> 172.21.15.131:0/2654203255 conn(0x7fca8809ba50 msgr2=0x7fca8809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:01.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.221+0000 7fca8ea00640 1 -- 172.21.15.131:0/2654203255 shutdown_connections 2024-04-28T12:39:01.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.221+0000 7fca8ea00640 1 -- 172.21.15.131:0/2654203255 wait complete. 2024-04-28T12:39:01.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.223+0000 7fca8ea00640 1 Processor -- start 2024-04-28T12:39:01.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.223+0000 7fca8ea00640 1 -- start start 2024-04-28T12:39:01.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.224+0000 7fca8ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8806e3b0 0x7fca881f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:01.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.224+0000 7fca8ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8815f050 0x7fca88155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:01.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.224+0000 7fca8ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca8815f430 0x7fca880ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:01.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.224+0000 7fca8ea00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fca880d0dc0 con 0x7fca8806e3b0 2024-04-28T12:39:01.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.224+0000 7fca8ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca880d1c90 con 0x7fca8815f430 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.224+0000 7fca8ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fca880cf5b0 con 0x7fca8815f050 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.224+0000 7fca87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8806e3b0 0x7fca881f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.224+0000 7fca87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8806e3b0 0x7fca881f8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47980/0 (socket says 172.21.15.131:47980) 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.224+0000 7fca87e00640 1 -- 172.21.15.131:0/4228876506 learned_addr learned my addr 172.21.15.131:0/4228876506 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.225+0000 7fca8ca00640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca8815f430 0x7fca880ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.225+0000 7fca87400640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8815f050 0x7fca88155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.225+0000 7fca87e00640 1 -- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8815f050 msgr2=0x7fca88155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.225+0000 7fca87e00640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8815f050 0x7fca88155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.225+0000 7fca87e00640 1 -- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca8815f430 msgr2=0x7fca880ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.225+0000 7fca87e00640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca8815f430 0x7fca880ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.225+0000 7fca87e00640 1 -- 172.21.15.131:0/4228876506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca881f8940 con 0x7fca8806e3b0 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.225+0000 7fca87400640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8815f050 0x7fca88155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.225+0000 7fca8ca00640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca8815f430 0x7fca880ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.225+0000 7fca87e00640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8806e3b0 0x7fca881f8400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fca7802a220 tx=0x7fca780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:01.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.226+0000 7fca7fe00640 1 -- 172.21.15.131:0/4228876506 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78037040 con 0x7fca8806e3b0 2024-04-28T12:39:01.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.226+0000 7fca7fe00640 1 -- 172.21.15.131:0/4228876506 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca78042070 con 0x7fca8806e3b0 2024-04-28T12:39:01.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.226+0000 7fca7fe00640 1 -- 172.21.15.131:0/4228876506 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78030bb0 con 0x7fca8806e3b0 2024-04-28T12:39:01.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.226+0000 7fca8ea00640 1 -- 172.21.15.131:0/4228876506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca88155a40 con 0x7fca8806e3b0 2024-04-28T12:39:01.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.226+0000 7fca8ea00640 1 -- 172.21.15.131:0/4228876506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca881f8ea0 con 0x7fca8806e3b0 2024-04-28T12:39:01.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.227+0000 7fca8ea00640 1 -- 172.21.15.131:0/4228876506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca880b5920 con 0x7fca8806e3b0 2024-04-28T12:39:01.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.228+0000 7fca7fe00640 1 -- 172.21.15.131:0/4228876506 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca78047460 con 0x7fca8806e3b0 2024-04-28T12:39:01.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.229+0000 7fca7fe00640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fca6406c920 0x7fca6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:01.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.229+0000 7fca7fe00640 1 -- 172.21.15.131:0/4228876506 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7fca780ac8f0 con 0x7fca8806e3b0 2024-04-28T12:39:01.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.230+0000 7fca87400640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fca6406c920 0x7fca6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:01.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.231+0000 7fca87400640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fca6406c920 0x7fca6406ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fca8815db30 tx=0x7fca70008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:01.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.233+0000 7fca7fe00640 1 -- 172.21.15.131:0/4228876506 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca88155a40 con 0x7fca8806e3b0 2024-04-28T12:39:01.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:01.435+0000 7fca8ea00640 1 -- 172.21.15.131:0/4228876506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fca88069350 con 0x7fca8806e3b0 2024-04-28T12:39:01.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.000+0000 7fca7fe00640 1 -- 172.21.15.131:0/4228876506 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7fca881f8940 con 0x7fca8806e3b0 2024-04-28T12:39:01.997 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data_ec' removed 2024-04-28T12:39:02.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.003+0000 7fca8ea00640 1 -- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fca6406c920 msgr2=0x7fca6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:02.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.003+0000 7fca8ea00640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fca6406c920 0x7fca6406ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fca8815db30 tx=0x7fca70008040 comp rx=0 tx=0).stop 2024-04-28T12:39:02.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.003+0000 7fca8ea00640 1 -- 172.21.15.131:0/4228876506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8806e3b0 msgr2=0x7fca881f8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:02.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.003+0000 7fca8ea00640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8806e3b0 0x7fca881f8400 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fca7802a220 tx=0x7fca780046c0 comp rx=0 tx=0).stop 2024-04-28T12:39:02.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.003+0000 7fca8ea00640 1 -- 172.21.15.131:0/4228876506 shutdown_connections 2024-04-28T12:39:02.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.003+0000 7fca8ea00640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fca6406c920 0x7fca6406ede0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:02.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.003+0000 7fca8ea00640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca8815f430 0x7fca880ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:02.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.003+0000 7fca8ea00640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8815f050 0x7fca88155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:02.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.003+0000 7fca8ea00640 1 --2- 172.21.15.131:0/4228876506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca8806e3b0 0x7fca881f8400 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:02.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.004+0000 7fca8ea00640 1 -- 172.21.15.131:0/4228876506 >> 172.21.15.131:0/4228876506 conn(0x7fca8809ba50 msgr2=0x7fca880b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:02.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.004+0000 7fca8ea00640 1 -- 172.21.15.131:0/4228876506 shutdown_connections 2024-04-28T12:39:02.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.004+0000 7fca8ea00640 1 -- 172.21.15.131:0/4228876506 wait complete. 2024-04-28T12:39:02.025 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-28T12:39:02.025 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-28T12:39:02.026 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-28T12:39:02.026 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:39:02.026 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-28T12:39:02.026 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-28T12:39:02.026 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-28T12:39:02.026 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:39:02.027 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-28T12:39:02.027 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-28T12:39:02.027 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-28T12:39:02.027 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:39:02.027 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-28T12:39:02.027 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-28T12:39:02.027 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-28T12:39:02.027 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:39:02.049 INFO:tasks.ceph.mds.a:Stopped 2024-04-28T12:39:02.049 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-28T12:39:02.050 INFO:tasks.ceph.mds.c:Stopped 2024-04-28T12:39:02.050 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-28T12:39:02.051 INFO:tasks.ceph.mds.d:Stopped 2024-04-28T12:39:02.051 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-28T12:39:02.052 INFO:tasks.ceph.mds.b:Stopped 2024-04-28T12:39:02.052 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-28T12:39:02.097 INFO:tasks.ceph.mds.a:Started 2024-04-28T12:39:02.097 DEBUG:teuthology.parallel:result is None 2024-04-28T12:39:02.097 INFO:tasks.ceph.mds.d:Started 2024-04-28T12:39:02.098 DEBUG:teuthology.parallel:result is None 2024-04-28T12:39:02.101 INFO:tasks.ceph.mds.b:Started 2024-04-28T12:39:02.101 DEBUG:teuthology.parallel:result is None 2024-04-28T12:39:02.102 INFO:tasks.ceph.mds.c:Started 2024-04-28T12:39:02.102 DEBUG:teuthology.parallel:result is None 2024-04-28T12:39:02.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-28T12:39:02.249 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-28T12:39:02.249 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-04-28T12:39:02.262 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-04-28T12:39:02.265 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-04-28T12:39:02.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.308+0000 7f051f400640 1 Processor -- start 2024-04-28T12:39:02.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.308+0000 7f051f400640 1 -- start start 2024-04-28T12:39:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.309+0000 7f051f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0518145590 0x7f05180ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.309+0000 7f051f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05180b3490 0x7f051806f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f051f400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05180cf060 con 0x7f051806f560 2024-04-28T12:39:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f051f400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f05180d0100 con 0x7f0518145590 2024-04-28T12:39:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f051f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f05180cf340 con 0x7f05180b3490 2024-04-28T12:39:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f051ca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0518145590 0x7f05180ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f051d400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f051806f560 0x7f05180d0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38118/0 (socket says 172.21.15.131:38118) 2024-04-28T12:39:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f051d400640 1 -- 172.21.15.131:0/4133245591 learned_addr learned my addr 172.21.15.131:0/4133245591 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f050fe00640 1 --2- 172.21.15.131:0/4133245591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05180b3490 0x7f051806f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f0517e00640 1 -- 172.21.15.131:0/4133245591 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 137686023 0 0) 0x7f05180cf060 con 0x7f051806f560 2024-04-28T12:39:02.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f0517e00640 1 -- 172.21.15.131:0/4133245591 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04fc003520 con 0x7f051806f560 2024-04-28T12:39:02.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f051ca00640 1 -- 172.21.15.131:0/4133245591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05180b3490 msgr2=0x7f051806f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:02.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f051ca00640 1 --2- 172.21.15.131:0/4133245591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05180b3490 0x7f051806f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:02.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f051ca00640 1 -- 172.21.15.131:0/4133245591 >> v1:172.21.15.131:6789/0 conn(0x7f051806f560 legacy=0x7f05180d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:02.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.310+0000 7f051ca00640 1 -- 172.21.15.131:0/4133245591 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05180cf620 con 0x7f0518145590 2024-04-28T12:39:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.312+0000 7f051ca00640 1 --2- 172.21.15.131:0/4133245591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0518145590 0x7f05180ce780 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f05080097a0 tx=0x7f050802c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96db3f27cce8b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.312+0000 7f0517e00640 1 -- 172.21.15.131:0/4133245591 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508039070 con 0x7f0518145590 2024-04-28T12:39:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.312+0000 7f0517e00640 1 -- 172.21.15.131:0/4133245591 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0508034640 con 0x7f0518145590 2024-04-28T12:39:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.312+0000 7f0517e00640 1 -- 172.21.15.131:0/4133245591 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508034a10 con 0x7f0518145590 2024-04-28T12:39:02.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.312+0000 7f051f400640 1 -- 172.21.15.131:0/4133245591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0518145590 msgr2=0x7f05180ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.312+0000 7f051f400640 1 --2- 172.21.15.131:0/4133245591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0518145590 0x7f05180ce780 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f05080097a0 tx=0x7f050802c390 comp rx=0 tx=0).stop 2024-04-28T12:39:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.312+0000 7f051f400640 1 -- 172.21.15.131:0/4133245591 shutdown_connections 2024-04-28T12:39:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.312+0000 7f051f400640 1 --2- 172.21.15.131:0/4133245591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05180b3490 0x7f051806f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.312+0000 7f051f400640 1 --2- 172.21.15.131:0/4133245591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0518145590 0x7f05180ce780 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.312+0000 7f051f400640 1 -- 172.21.15.131:0/4133245591 >> 172.21.15.131:0/4133245591 conn(0x7f05180b2900 msgr2=0x7f05180b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.313+0000 7f051f400640 1 -- 172.21.15.131:0/4133245591 shutdown_connections 2024-04-28T12:39:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.313+0000 7f051f400640 1 -- 172.21.15.131:0/4133245591 wait complete. 2024-04-28T12:39:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.314+0000 7f051f400640 1 Processor -- start 2024-04-28T12:39:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.314+0000 7f051f400640 1 -- start start 2024-04-28T12:39:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.315+0000 7f051f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f051806f560 0x7f051815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.315+0000 7f051f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05180b3490 0x7f05181e5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.315+0000 7f051f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0518145590 0x7f051815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.315+0000 7f051f400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0518145970 con 0x7f0518145590 2024-04-28T12:39:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.315+0000 7f051f400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05180cfbe0 con 0x7f051806f560 2024-04-28T12:39:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.315+0000 7f051f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f05180cecc0 con 0x7f05180b3490 2024-04-28T12:39:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.316+0000 7f051d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0518145590 0x7f051815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.316+0000 7f051d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0518145590 0x7f051815eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48040/0 (socket says 172.21.15.131:48040) 2024-04-28T12:39:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.316+0000 7f051d400640 1 -- 172.21.15.131:0/3170403643 learned_addr learned my addr 172.21.15.131:0/3170403643 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.316+0000 7f050fe00640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05180b3490 0x7f05181e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.316+0000 7f051ca00640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f051806f560 0x7f051815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.316+0000 7f051d400640 1 -- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05180b3490 msgr2=0x7f05181e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.316+0000 7f051d400640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05180b3490 0x7f05181e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:02.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.316+0000 7f051d400640 1 -- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f051806f560 msgr2=0x7f051815b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:02.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.316+0000 7f051d400640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f051806f560 0x7f051815b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:02.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.316+0000 7f051d400640 1 -- 172.21.15.131:0/3170403643 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05181e5660 con 0x7f0518145590 2024-04-28T12:39:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.317+0000 7f051d400640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0518145590 0x7f051815eac0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f051000cb40 tx=0x7f051000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.317+0000 7f0515600640 1 -- 172.21.15.131:0/3170403643 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510017070 con 0x7f0518145590 2024-04-28T12:39:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.317+0000 7f0515600640 1 -- 172.21.15.131:0/3170403643 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051000aca0 con 0x7f0518145590 2024-04-28T12:39:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.317+0000 7f051f400640 1 -- 172.21.15.131:0/3170403643 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051815da20 con 0x7f0518145590 2024-04-28T12:39:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.317+0000 7f0515600640 1 -- 172.21.15.131:0/3170403643 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510012620 con 0x7f0518145590 2024-04-28T12:39:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.317+0000 7f051f400640 1 -- 172.21.15.131:0/3170403643 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051815dfe0 con 0x7f0518145590 2024-04-28T12:39:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.318+0000 7f051f400640 1 -- 172.21.15.131:0/3170403643 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0518069350 con 0x7f0518145590 2024-04-28T12:39:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.319+0000 7f0515600640 1 -- 172.21.15.131:0/3170403643 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f051001a070 con 0x7f0518145590 2024-04-28T12:39:02.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.321+0000 7f0515600640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f04f406c920 0x7f04f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:02.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.321+0000 7f0515600640 1 -- 172.21.15.131:0/3170403643 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7f051008b800 con 0x7f0518145590 2024-04-28T12:39:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.321+0000 7f051ca00640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f04f406c920 0x7f04f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.322+0000 7f051ca00640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f04f406c920 0x7f04f406ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0508036000 tx=0x7f0508030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.325+0000 7f0515600640 1 -- 172.21.15.131:0/3170403643 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05181e5660 con 0x7f0518145590 2024-04-28T12:39:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:02.539+0000 7f051f400640 1 -- 172.21.15.131:0/3170403643 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0518144110 con 0x7f0518145590 2024-04-28T12:39:03.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.023+0000 7f0515600640 1 -- 172.21.15.131:0/3170403643 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7f05100596b0 con 0x7f0518145590 2024-04-28T12:39:03.020 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-04-28T12:39:03.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.026+0000 7f051f400640 1 -- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f04f406c920 msgr2=0x7f04f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.027+0000 7f051f400640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f04f406c920 0x7f04f406ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0508036000 tx=0x7f0508030000 comp rx=0 tx=0).stop 2024-04-28T12:39:03.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.027+0000 7f051f400640 1 -- 172.21.15.131:0/3170403643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0518145590 msgr2=0x7f051815eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.027+0000 7f051f400640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0518145590 0x7f051815eac0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f051000cb40 tx=0x7f051000a060 comp rx=0 tx=0).stop 2024-04-28T12:39:03.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.027+0000 7f051f400640 1 -- 172.21.15.131:0/3170403643 shutdown_connections 2024-04-28T12:39:03.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.027+0000 7f051f400640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f04f406c920 0x7f04f406ede0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.027+0000 7f051f400640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0518145590 0x7f051815eac0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.027+0000 7f051f400640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05180b3490 0x7f05181e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.027+0000 7f051f400640 1 --2- 172.21.15.131:0/3170403643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f051806f560 0x7f051815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.027+0000 7f051f400640 1 -- 172.21.15.131:0/3170403643 >> 172.21.15.131:0/3170403643 conn(0x7f05180b2900 msgr2=0x7f05181435f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:03.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.027+0000 7f051f400640 1 -- 172.21.15.131:0/3170403643 shutdown_connections 2024-04-28T12:39:03.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.027+0000 7f051f400640 1 -- 172.21.15.131:0/3170403643 wait complete. 2024-04-28T12:39:03.046 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-28T12:39:03.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.248+0000 7fef5fe00640 1 Processor -- start 2024-04-28T12:39:03.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.248+0000 7fef5fe00640 1 -- start start 2024-04-28T12:39:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.250+0000 7fef5fe00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fef601463e0 0x7fef600b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.250+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60161460 0x7fef60160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.250+0000 7fef5fe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef600b50e0 con 0x7fef60160d90 2024-04-28T12:39:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.250+0000 7fef5fe00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fef601611e0 con 0x7fef601463e0 2024-04-28T12:39:03.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.250+0000 7fef5fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fef600b4560 con 0x7fef60161460 2024-04-28T12:39:03.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.250+0000 7fef5f400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fef60160d90 0x7fef600cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38122/0 (socket says 172.21.15.131:38122) 2024-04-28T12:39:03.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.250+0000 7fef5f400640 1 -- 172.21.15.131:0/3883976240 learned_addr learned my addr 172.21.15.131:0/3883976240 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:03.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5e000640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60161460 0x7fef60160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5ea00640 1 --2- 172.21.15.131:0/3883976240 >> v2:172.21.15.157:3300/0 conn(0x7fef601463e0 0x7fef600b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4264192900 0 0) 0x7fef600b50e0 con 0x7fef60160d90 2024-04-28T12:39:03.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef44003520 con 0x7fef60160d90 2024-04-28T12:39:03.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2924261225 0 0) 0x7fef44003520 con 0x7fef60160d90 2024-04-28T12:39:03.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60161460 msgr2=0x7fef60160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60161460 0x7fef60160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 >> v2:172.21.15.157:3300/0 conn(0x7fef601463e0 msgr2=0x7fef600b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 --2- 172.21.15.131:0/3883976240 >> v2:172.21.15.157:3300/0 conn(0x7fef601463e0 0x7fef600b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef600b4840 con 0x7fef60160d90 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fef58001670 con 0x7fef60160d90 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 >> v1:172.21.15.131:6789/0 conn(0x7fef60160d90 legacy=0x7fef600cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef440039b0 0x7fef44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef44004800 0x7fef44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60161460 0x7fef440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fef601611e0 con 0x7fef440039b0 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef600b4560 con 0x7fef44004800 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.251+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fef4400ba10 con 0x7fef60161460 2024-04-28T12:39:03.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.252+0000 7fef5ea00640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef440039b0 0x7fef44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.252+0000 7fef5e000640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef44004800 0x7fef44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.252+0000 7fef5f400640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60161460 0x7fef440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.252+0000 7fef5ea00640 1 -- 172.21.15.131:0/3883976240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60161460 msgr2=0x7fef440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:03.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.252+0000 7fef5ea00640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60161460 0x7fef440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.252+0000 7fef5ea00640 1 -- 172.21.15.131:0/3883976240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef44004800 msgr2=0x7fef44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:03.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.252+0000 7fef5ea00640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef44004800 0x7fef44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.252+0000 7fef5ea00640 1 -- 172.21.15.131:0/3883976240 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef44003520 con 0x7fef440039b0 2024-04-28T12:39:03.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.252+0000 7fef5e000640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef44004800 0x7fef44004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:03.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.254+0000 7fef5ea00640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef440039b0 0x7fef44003dd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fef50009af0 tx=0x7fef44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5255a96922750e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:03.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.254+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50036070 con 0x7fef440039b0 2024-04-28T12:39:03.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.254+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef50031400 con 0x7fef440039b0 2024-04-28T12:39:03.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.254+0000 7fef5d600640 1 -- 172.21.15.131:0/3883976240 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50009690 con 0x7fef440039b0 2024-04-28T12:39:03.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.255+0000 7fef5fe00640 1 -- 172.21.15.131:0/3883976240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef440039b0 msgr2=0x7fef44003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.255+0000 7fef5fe00640 1 --2- 172.21.15.131:0/3883976240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef440039b0 0x7fef44003dd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fef50009af0 tx=0x7fef44005830 comp rx=0 tx=0).stop 2024-04-28T12:39:03.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.255+0000 7fef5f400640 1 -- 172.21.15.131:0/3883976240 reap_dead start 2024-04-28T12:39:03.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.255+0000 7fef5fe00640 1 -- 172.21.15.131:0/3883976240 shutdown_connections 2024-04-28T12:39:03.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.255+0000 7fef5fe00640 1 -- 172.21.15.131:0/3883976240 >> 172.21.15.131:0/3883976240 conn(0x7fef600b26b0 msgr2=0x7fef600b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:03.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.255+0000 7fef5fe00640 1 -- 172.21.15.131:0/3883976240 shutdown_connections 2024-04-28T12:39:03.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.255+0000 7fef5fe00640 1 -- 172.21.15.131:0/3883976240 wait complete. 2024-04-28T12:39:03.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.257+0000 7fef5fe00640 1 Processor -- start 2024-04-28T12:39:03.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.257+0000 7fef5fe00640 1 -- start start 2024-04-28T12:39:03.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.258+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef440039b0 0x7fef601453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.258+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60145900 0x7fef600aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.258+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef600ab1e0 0x7fef600ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.258+0000 7fef5fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fef601467c0 con 0x7fef440039b0 2024-04-28T12:39:03.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.258+0000 7fef5fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef600b4e00 con 0x7fef600ab1e0 2024-04-28T12:39:03.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.258+0000 7fef5fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fef600b5df0 con 0x7fef60145900 2024-04-28T12:39:03.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.259+0000 7fef5ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef440039b0 0x7fef601453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.259+0000 7fef5ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef440039b0 0x7fef601453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48048/0 (socket says 172.21.15.131:48048) 2024-04-28T12:39:03.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.259+0000 7fef5ea00640 1 -- 172.21.15.131:0/430854833 learned_addr learned my addr 172.21.15.131:0/430854833 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:03.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.259+0000 7fef5e000640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60145900 0x7fef600aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.259+0000 7fef5f400640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef600ab1e0 0x7fef600ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.259+0000 7fef5ea00640 1 -- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60145900 msgr2=0x7fef600aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.259+0000 7fef5ea00640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60145900 0x7fef600aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.259+0000 7fef5ea00640 1 -- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef600ab1e0 msgr2=0x7fef600ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.259+0000 7fef5ea00640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef600ab1e0 0x7fef600ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.259+0000 7fef5ea00640 1 -- 172.21.15.131:0/430854833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef601426b0 con 0x7fef440039b0 2024-04-28T12:39:03.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.259+0000 7fef5f400640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef600ab1e0 0x7fef600ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:03.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.260+0000 7fef5ea00640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef440039b0 0x7fef601453c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fef50009c10 tx=0x7fef50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:03.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.260+0000 7fef57400640 1 -- 172.21.15.131:0/430854833 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50041020 con 0x7fef440039b0 2024-04-28T12:39:03.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.260+0000 7fef57400640 1 -- 172.21.15.131:0/430854833 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5002ade0 con 0x7fef440039b0 2024-04-28T12:39:03.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.260+0000 7fef57400640 1 -- 172.21.15.131:0/430854833 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50036040 con 0x7fef440039b0 2024-04-28T12:39:03.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.260+0000 7fef5fe00640 1 -- 172.21.15.131:0/430854833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef60142990 con 0x7fef440039b0 2024-04-28T12:39:03.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.260+0000 7fef5fe00640 1 -- 172.21.15.131:0/430854833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef60142f50 con 0x7fef440039b0 2024-04-28T12:39:03.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.261+0000 7fef5fe00640 1 -- 172.21.15.131:0/430854833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef60069350 con 0x7fef440039b0 2024-04-28T12:39:03.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.262+0000 7fef57400640 1 -- 172.21.15.131:0/430854833 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef50044020 con 0x7fef440039b0 2024-04-28T12:39:03.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.263+0000 7fef57400640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fef4006c890 0x7fef4006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.263+0000 7fef57400640 1 -- 172.21.15.131:0/430854833 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fef500abf40 con 0x7fef440039b0 2024-04-28T12:39:03.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.264+0000 7fef5e000640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fef4006c890 0x7fef4006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.265+0000 7fef5e000640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fef4006c890 0x7fef4006ed50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fef60073030 tx=0x7fef4c00d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:03.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.268+0000 7fef57400640 1 -- 172.21.15.131:0/430854833 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef5004b030 con 0x7fef440039b0 2024-04-28T12:39:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.530+0000 7fef5fe00640 1 -- 172.21.15.131:0/430854833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fef60073f40 con 0x7fef440039b0 2024-04-28T12:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.532+0000 7fef57400640 1 -- 172.21.15.131:0/430854833 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v5) ==== 80+0+6645 (secure 0 0 0) 0x7fef5007a490 con 0x7fef440039b0 2024-04-28T12:39:03.528 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:03.529 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:39:03.529 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-04-28T12:39:03.529 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.529 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-04-28T12:39:03.529 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDCQi5mAK7dOBAAPuF0VSQ8/oA/TabJenHHTA==", 2024-04-28T12:39:03.529 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.529 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-04-28T12:39:03.529 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-04-28T12:39:03.529 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-04-28T12:39:03.529 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.529 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.529 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.530 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.530 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-04-28T12:39:03.530 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDDQi5my+1kGxAAQYO+yf04zcg7fflJbD6cMg==", 2024-04-28T12:39:03.530 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.530 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-04-28T12:39:03.530 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-04-28T12:39:03.530 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-04-28T12:39:03.530 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.530 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.530 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.530 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDDQi5mn6mlDBAA4YLF/yZrzfJVuw7NuYNE6g==", 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDDQi5mejXpJhAAKu1RVfnAQb9/dZLira9nAA==", 2024-04-28T12:39:03.531 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDHQi5m1zB9KBAAA4XL4E3Zimv7IfK8gIwlog==", 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.532 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDKQi5mCqcTMxAAp2DAM3sQFnWvE7tUmDCsgg==", 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDOQi5mVYspEhAAIAcbyMkwxxHmRB2rTnYBZQ==", 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.533 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDRQi5mcOO/LBAACgiwbkFt6BTFeXDfh0RtsQ==", 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.534 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDWQi5mPxxOJhAAHZqlRUkEKZUvtitkoULnAg==", 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDZQi5mRgeuMRAAlPWR+At5yl/wIrWFvEtFRQ==", 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.535 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDcQi5mKH1ZOxAA+aQ7qRfEOfhAzgSjx3UQYg==", 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.536 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDgQi5mwlm9ChAA9oikwI0iUbxqbGXjXLZflw==", 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDDQi5mElI+MxAAW17tKD+nhz2A3X8a4qZmbw==", 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.537 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDEQi5mwR0tChAAJTudcvUJmCRLI5oijLUqOw==", 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-04-28T12:39:03.538 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDEQi5mSa75ABAAGkSnVFXcrqSGrH11c1Yy6w==", 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-04-28T12:39:03.539 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDEQi5mBMtmExAAe8U6grexkKUjuwlLlKN/wg==", 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDBQi5mTaCENhAAE+Zs1n7PlmCjxUIQnxjpiQ==", 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-04-28T12:39:03.540 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-mds", 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDtQi5meJXIMBAAZabFd6TLoJDlPZ5vCs97Fg==", 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-mds" 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-mgr", 2024-04-28T12:39:03.541 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDtQi5mx7HIMBAAPjyT9Y4zXyNV+/eFy65HKQ==", 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-osd", 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDtQi5mMcrIMBAAe3cryoXkNwrMdoGFAEEb8g==", 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-osd" 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.542 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rbd", 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDtQi5ml+LIMBAAgGu3rwfIBkzuSGGl0ec7FQ==", 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDtQi5mifvIMBAAF/gqNfuMJlLc8+Zse4Jylg==", 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.543 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rgw", 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDtQi5mgxrJMBAAhmmi9rdTaix6B9ioQLH5Gg==", 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDCQi5mlcRqLxAArJ9SXw5eNDzPr0nrWHUaPw==", 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:03.544 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:03.545 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-04-28T12:39:03.545 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDCQi5mPkqdJBAAYU06FiAGyuWtxBTcwsXUQg==", 2024-04-28T12:39:03.545 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-04-28T12:39:03.545 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-04-28T12:39:03.545 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-04-28T12:39:03.545 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-04-28T12:39:03.545 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.545 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:03.545 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-04-28T12:39:03.545 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:39:03.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.534+0000 7fef5fe00640 1 -- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fef4006c890 msgr2=0x7fef4006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.534+0000 7fef5fe00640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fef4006c890 0x7fef4006ed50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fef60073030 tx=0x7fef4c00d040 comp rx=0 tx=0).stop 2024-04-28T12:39:03.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.534+0000 7fef5fe00640 1 -- 172.21.15.131:0/430854833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef440039b0 msgr2=0x7fef601453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.534+0000 7fef5fe00640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef440039b0 0x7fef601453c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fef50009c10 tx=0x7fef50004730 comp rx=0 tx=0).stop 2024-04-28T12:39:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.534+0000 7fef5fe00640 1 -- 172.21.15.131:0/430854833 shutdown_connections 2024-04-28T12:39:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.534+0000 7fef5fe00640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fef4006c890 0x7fef4006ed50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.534+0000 7fef5fe00640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef600ab1e0 0x7fef600ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.534+0000 7fef5fe00640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef60145900 0x7fef600aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.534+0000 7fef5fe00640 1 --2- 172.21.15.131:0/430854833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef440039b0 0x7fef601453c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.534+0000 7fef5fe00640 1 -- 172.21.15.131:0/430854833 >> 172.21.15.131:0/430854833 conn(0x7fef600b26b0 msgr2=0x7fef600732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.534+0000 7fef5fe00640 1 -- 172.21.15.131:0/430854833 shutdown_connections 2024-04-28T12:39:03.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.535+0000 7fef5fe00640 1 -- 172.21.15.131:0/430854833 wait complete. 2024-04-28T12:39:03.549 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-28T12:39:03.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.748+0000 7f024b200640 1 Processor -- start 2024-04-28T12:39:03.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.748+0000 7f024b200640 1 -- start start 2024-04-28T12:39:03.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.750+0000 7f024b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c15b900 0x7f024c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.750+0000 7f024b200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f024c071d20 con 0x7f024c15feb0 2024-04-28T12:39:03.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.750+0000 7f024b200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f024c0b4f00 con 0x7f024c06f840 2024-04-28T12:39:03.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.750+0000 7f024b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f024c0b51e0 con 0x7f024c15b900 2024-04-28T12:39:03.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.750+0000 7f024a800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f024c15feb0 0x7f024c15bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38128/0 (socket says 172.21.15.131:38128) 2024-04-28T12:39:03.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.750+0000 7f024a800640 1 -- 172.21.15.131:0/1884825779 learned_addr learned my addr 172.21.15.131:0/1884825779 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:03.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.751+0000 7f0249e00640 1 --2- 172.21.15.131:0/1884825779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c15b900 0x7f024c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.751+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2784704141 0 0) 0x7f024c071d20 con 0x7f024c15feb0 2024-04-28T12:39:03.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.751+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0230003520 con 0x7f024c15feb0 2024-04-28T12:39:03.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.751+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2073280195 0 0) 0x7f0230003520 con 0x7f024c15feb0 2024-04-28T12:39:03.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.751+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c15b900 msgr2=0x7f024c0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:03.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.751+0000 7f0249400640 1 --2- 172.21.15.131:0/1884825779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c15b900 0x7f024c0b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.751+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 >> v1:172.21.15.157:6789/0 conn(0x7f024c06f840 legacy=0x7f024c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.751+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024c071f90 con 0x7f024c15feb0 2024-04-28T12:39:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.751+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f02440019f0 con 0x7f024c15feb0 2024-04-28T12:39:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.751+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 >> v1:172.21.15.131:6789/0 conn(0x7f024c15feb0 legacy=0x7f024c15bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.751+0000 7f0249400640 1 --2- 172.21.15.131:0/1884825779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02300039e0 0x7f0230003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.751+0000 7f0249400640 1 --2- 172.21.15.131:0/1884825779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c15b900 0x7f0230004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.752+0000 7f0249400640 1 --2- 172.21.15.131:0/1884825779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0230004d40 0x7f02300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.752+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f024c0b51e0 con 0x7f0230004d40 2024-04-28T12:39:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.752+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f023000baa0 con 0x7f02300039e0 2024-04-28T12:39:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.752+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f023000deb0 con 0x7f024c15b900 2024-04-28T12:39:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.752+0000 7f024a800640 1 --2- 172.21.15.131:0/1884825779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0230004d40 0x7f02300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.752+0000 7f0241400640 1 --2- 172.21.15.131:0/1884825779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c15b900 0x7f0230004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.752+0000 7f0249e00640 1 --2- 172.21.15.131:0/1884825779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02300039e0 0x7f0230003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.752+0000 7f024a800640 1 -- 172.21.15.131:0/1884825779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c15b900 msgr2=0x7f0230004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:03.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.752+0000 7f024a800640 1 --2- 172.21.15.131:0/1884825779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c15b900 0x7f0230004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.752+0000 7f024a800640 1 -- 172.21.15.131:0/1884825779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02300039e0 msgr2=0x7f0230003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:03.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.752+0000 7f024a800640 1 --2- 172.21.15.131:0/1884825779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02300039e0 0x7f0230003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.752+0000 7f024a800640 1 -- 172.21.15.131:0/1884825779 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0230003520 con 0x7f0230004d40 2024-04-28T12:39:03.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.753+0000 7f024a800640 1 --2- 172.21.15.131:0/1884825779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0230004d40 0x7f02300051c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f024400a760 tx=0x7f0230005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c84928ff578502 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:03.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.753+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024400a3b0 con 0x7f0230004d40 2024-04-28T12:39:03.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.754+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024402f070 con 0x7f0230004d40 2024-04-28T12:39:03.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.754+0000 7f0249400640 1 -- 172.21.15.131:0/1884825779 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024402bbe0 con 0x7f0230004d40 2024-04-28T12:39:03.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.754+0000 7f024b200640 1 -- 172.21.15.131:0/1884825779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0230004d40 msgr2=0x7f02300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.754+0000 7f024b200640 1 --2- 172.21.15.131:0/1884825779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0230004d40 0x7f02300051c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f024400a760 tx=0x7f0230005830 comp rx=0 tx=0).stop 2024-04-28T12:39:03.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.754+0000 7f024a800640 1 -- 172.21.15.131:0/1884825779 reap_dead start 2024-04-28T12:39:03.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.754+0000 7f024b200640 1 -- 172.21.15.131:0/1884825779 shutdown_connections 2024-04-28T12:39:03.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.754+0000 7f024b200640 1 -- 172.21.15.131:0/1884825779 >> 172.21.15.131:0/1884825779 conn(0x7f024c0b26b0 msgr2=0x7f024c0b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:03.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.754+0000 7f024b200640 1 -- 172.21.15.131:0/1884825779 shutdown_connections 2024-04-28T12:39:03.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.754+0000 7f024b200640 1 -- 172.21.15.131:0/1884825779 wait complete. 2024-04-28T12:39:03.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.756+0000 7f024b200640 1 Processor -- start 2024-04-28T12:39:03.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.756+0000 7f024b200640 1 -- start start 2024-04-28T12:39:03.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.757+0000 7f024b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0230004d40 0x7f024c140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.757+0000 7f024b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c1591e0 0x7f024c159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.757+0000 7f024b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f024c1564d0 0x7f024c156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.757+0000 7f024b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f024c073640 con 0x7f0230004d40 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.757+0000 7f024b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f024c0734c0 con 0x7f024c1564d0 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.757+0000 7f024b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f024c0737c0 con 0x7f024c1591e0 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.757+0000 7f0249e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0230004d40 0x7f024c140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.757+0000 7f0249e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0230004d40 0x7f024c140170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48072/0 (socket says 172.21.15.131:48072) 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.757+0000 7f0249e00640 1 -- 172.21.15.131:0/2047405377 learned_addr learned my addr 172.21.15.131:0/2047405377 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f0241400640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c1591e0 0x7f024c159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f024a800640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f024c1564d0 0x7f024c156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f0249e00640 1 -- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c1591e0 msgr2=0x7f024c159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f0249e00640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c1591e0 0x7f024c159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f0249e00640 1 -- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f024c1564d0 msgr2=0x7f024c156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f0249e00640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f024c1564d0 0x7f024c156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f0249e00640 1 -- 172.21.15.131:0/2047405377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024c14b530 con 0x7f0230004d40 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f0249e00640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0230004d40 0x7f024c140170 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f023c010890 tx=0x7f023c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:03.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f0242a00640 1 -- 172.21.15.131:0/2047405377 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c00f040 con 0x7f0230004d40 2024-04-28T12:39:03.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f0242a00640 1 -- 172.21.15.131:0/2047405377 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f023c00eb00 con 0x7f0230004d40 2024-04-28T12:39:03.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f0242a00640 1 -- 172.21.15.131:0/2047405377 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c01c400 con 0x7f0230004d40 2024-04-28T12:39:03.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f024b200640 1 -- 172.21.15.131:0/2047405377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f024c157610 con 0x7f0230004d40 2024-04-28T12:39:03.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.758+0000 7f024b200640 1 -- 172.21.15.131:0/2047405377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f024c157bd0 con 0x7f0230004d40 2024-04-28T12:39:03.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.759+0000 7f024b200640 1 -- 172.21.15.131:0/2047405377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f024c069350 con 0x7f0230004d40 2024-04-28T12:39:03.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.760+0000 7f0242a00640 1 -- 172.21.15.131:0/2047405377 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f023c01c5a0 con 0x7f0230004d40 2024-04-28T12:39:03.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.762+0000 7f0242a00640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f022806c8f0 0x7f022806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:03.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.762+0000 7f0242a00640 1 -- 172.21.15.131:0/2047405377 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f023c08bfb0 con 0x7f0230004d40 2024-04-28T12:39:03.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.762+0000 7f0241400640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f022806c8f0 0x7f022806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:03.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.764+0000 7f0241400640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f022806c8f0 0x7f022806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0234007fd0 tx=0x7f0234009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:03.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:03.766+0000 7f0242a00640 1 -- 172.21.15.131:0/2047405377 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f023c018030 con 0x7f0230004d40 2024-04-28T12:39:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.023+0000 7f024b200640 1 -- 172.21.15.131:0/2047405377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f024c155fb0 con 0x7f0230004d40 2024-04-28T12:39:04.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.033+0000 7f0242a00640 1 -- 172.21.15.131:0/2047405377 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f023c05a5a0 con 0x7f0230004d40 2024-04-28T12:39:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.036+0000 7f024b200640 1 -- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f022806c8f0 msgr2=0x7f022806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.037+0000 7f024b200640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f022806c8f0 0x7f022806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0234007fd0 tx=0x7f0234009040 comp rx=0 tx=0).stop 2024-04-28T12:39:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.037+0000 7f024b200640 1 -- 172.21.15.131:0/2047405377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0230004d40 msgr2=0x7f024c140170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.037+0000 7f024b200640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0230004d40 0x7f024c140170 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f023c010890 tx=0x7f023c010d50 comp rx=0 tx=0).stop 2024-04-28T12:39:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.037+0000 7f024b200640 1 -- 172.21.15.131:0/2047405377 shutdown_connections 2024-04-28T12:39:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.037+0000 7f024b200640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f022806c8f0 0x7f022806edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.037+0000 7f024b200640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f024c1564d0 0x7f024c156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.037+0000 7f024b200640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f024c1591e0 0x7f024c159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.037+0000 7f024b200640 1 --2- 172.21.15.131:0/2047405377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0230004d40 0x7f024c140170 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.037+0000 7f024b200640 1 -- 172.21.15.131:0/2047405377 >> 172.21.15.131:0/2047405377 conn(0x7f024c0b26b0 msgr2=0x7f024c06e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.037+0000 7f024b200640 1 -- 172.21.15.131:0/2047405377 shutdown_connections 2024-04-28T12:39:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.037+0000 7f024b200640 1 -- 172.21.15.131:0/2047405377 wait complete. 2024-04-28T12:39:04.053 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-28T12:39:04.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.242+0000 7f4e95e00640 1 Processor -- start 2024-04-28T12:39:04.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.243+0000 7f4e95e00640 1 -- start start 2024-04-28T12:39:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.244+0000 7f4e95e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4e90160ae0 0x7f4e9015df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.244+0000 7f4e95e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e900d0da0 0x7f4e900d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.244+0000 7f4e95e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e90161170 con 0x7f4e9013fc90 2024-04-28T12:39:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.244+0000 7f4e95e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4e90160f50 con 0x7f4e90160ae0 2024-04-28T12:39:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.244+0000 7f4e95e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4e900d1750 con 0x7f4e900d0da0 2024-04-28T12:39:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.244+0000 7f4e8e000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4e90160ae0 0x7f4e9015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.244+0000 7f4e8e000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4e90160ae0 0x7f4e9015df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48088/0 (socket says 172.21.15.131:48088) 2024-04-28T12:39:04.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.244+0000 7f4e8e000640 1 -- 172.21.15.131:0/16159501 learned_addr learned my addr 172.21.15.131:0/16159501 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:04.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.245+0000 7f4e8f400640 1 --2- 172.21.15.131:0/16159501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e900d0da0 0x7f4e900d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:04.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.245+0000 7f4e8e000640 1 -- 172.21.15.131:0/16159501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e900d0da0 msgr2=0x7f4e900d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:04.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.245+0000 7f4e8e000640 1 --2- 172.21.15.131:0/16159501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e900d0da0 0x7f4e900d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.245+0000 7f4e8e000640 1 -- 172.21.15.131:0/16159501 >> v1:172.21.15.157:6789/0 conn(0x7f4e9013fc90 legacy=0x7f4e90142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:04.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.245+0000 7f4e8e000640 1 -- 172.21.15.131:0/16159501 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e90161450 con 0x7f4e90160ae0 2024-04-28T12:39:04.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.246+0000 7f4e8e000640 1 --2- 172.21.15.131:0/16159501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e90160ae0 0x7f4e9015df70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009870 tx=0x7f4e7802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b11a75f7bc1f62d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:04.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.246+0000 7f4e8d600640 1 -- 172.21.15.131:0/16159501 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78037070 con 0x7f4e90160ae0 2024-04-28T12:39:04.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.246+0000 7f4e8d600640 1 -- 172.21.15.131:0/16159501 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e78032440 con 0x7f4e90160ae0 2024-04-28T12:39:04.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.246+0000 7f4e8d600640 1 -- 172.21.15.131:0/16159501 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78032810 con 0x7f4e90160ae0 2024-04-28T12:39:04.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.247+0000 7f4e95e00640 1 -- 172.21.15.131:0/16159501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e90160ae0 msgr2=0x7f4e9015df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:04.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.247+0000 7f4e95e00640 1 --2- 172.21.15.131:0/16159501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e90160ae0 0x7f4e9015df70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009870 tx=0x7f4e7802a210 comp rx=0 tx=0).stop 2024-04-28T12:39:04.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.247+0000 7f4e95e00640 1 -- 172.21.15.131:0/16159501 shutdown_connections 2024-04-28T12:39:04.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.247+0000 7f4e95e00640 1 --2- 172.21.15.131:0/16159501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e900d0da0 0x7f4e900d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.247+0000 7f4e95e00640 1 --2- 172.21.15.131:0/16159501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e90160ae0 0x7f4e9015df70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.247+0000 7f4e95e00640 1 -- 172.21.15.131:0/16159501 >> 172.21.15.131:0/16159501 conn(0x7f4e900682a0 msgr2=0x7f4e900686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:04.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.247+0000 7f4e95e00640 1 -- 172.21.15.131:0/16159501 shutdown_connections 2024-04-28T12:39:04.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.247+0000 7f4e95e00640 1 -- 172.21.15.131:0/16159501 wait complete. 2024-04-28T12:39:04.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.249+0000 7f4e95e00640 1 Processor -- start 2024-04-28T12:39:04.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.249+0000 7f4e95e00640 1 -- start start 2024-04-28T12:39:04.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.250+0000 7f4e95e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e900d0da0 0x7f4e9006ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:04.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.250+0000 7f4e95e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e9013fc90 0x7f4e9015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:04.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.250+0000 7f4e95e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e90160ae0 0x7f4e9015c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.250+0000 7f4e95e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90142770 con 0x7f4e90160ae0 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.250+0000 7f4e95e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e901433a0 con 0x7f4e9013fc90 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.250+0000 7f4e95e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4e90143520 con 0x7f4e900d0da0 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.250+0000 7f4e8f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e90160ae0 0x7f4e9015c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.250+0000 7f4e8f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e90160ae0 0x7f4e9015c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48092/0 (socket says 172.21.15.131:48092) 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.250+0000 7f4e8f400640 1 -- 172.21.15.131:0/341810820 learned_addr learned my addr 172.21.15.131:0/341810820 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.251+0000 7f4e8ea00640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e900d0da0 0x7f4e9006ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.251+0000 7f4e8e000640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e9013fc90 0x7f4e9015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.251+0000 7f4e8f400640 1 -- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e900d0da0 msgr2=0x7f4e9006ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.251+0000 7f4e8f400640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e900d0da0 0x7f4e9006ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.251+0000 7f4e8f400640 1 -- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e9013fc90 msgr2=0x7f4e9015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.251+0000 7f4e8f400640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e9013fc90 0x7f4e9015bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.251+0000 7f4e8f400640 1 -- 172.21.15.131:0/341810820 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9006e840 con 0x7f4e90160ae0 2024-04-28T12:39:04.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.251+0000 7f4e8f400640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e90160ae0 0x7f4e9015c470 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4e88007e40 tx=0x7f4e8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:04.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.251+0000 7f4e87400640 1 -- 172.21.15.131:0/341810820 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e88014620 con 0x7f4e90160ae0 2024-04-28T12:39:04.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.251+0000 7f4e87400640 1 -- 172.21.15.131:0/341810820 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e88019070 con 0x7f4e90160ae0 2024-04-28T12:39:04.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.251+0000 7f4e87400640 1 -- 172.21.15.131:0/341810820 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e88014a20 con 0x7f4e90160ae0 2024-04-28T12:39:04.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.251+0000 7f4e95e00640 1 -- 172.21.15.131:0/341810820 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9006eb20 con 0x7f4e90160ae0 2024-04-28T12:39:04.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.252+0000 7f4e95e00640 1 -- 172.21.15.131:0/341810820 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9006e410 con 0x7f4e90160ae0 2024-04-28T12:39:04.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.253+0000 7f4e95e00640 1 -- 172.21.15.131:0/341810820 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9015fa70 con 0x7f4e90160ae0 2024-04-28T12:39:04.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.254+0000 7f4e87400640 1 -- 172.21.15.131:0/341810820 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e88014bc0 con 0x7f4e90160ae0 2024-04-28T12:39:04.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.255+0000 7f4e87400640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4e6c06c8f0 0x7f4e6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:04.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.255+0000 7f4e87400640 1 -- 172.21.15.131:0/341810820 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f4e8808c2a0 con 0x7f4e90160ae0 2024-04-28T12:39:04.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.255+0000 7f4e8ea00640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4e6c06c8f0 0x7f4e6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:04.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.256+0000 7f4e8ea00640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4e6c06c8f0 0x7f4e6c06edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4e80003d90 tx=0x7f4e80009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:04.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.259+0000 7f4e87400640 1 -- 172.21.15.131:0/341810820 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e9015fa70 con 0x7f4e90160ae0 2024-04-28T12:39:04.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.547+0000 7f4e95e00640 1 -- 172.21.15.131:0/341810820 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f4e90069350 con 0x7f4e90160ae0 2024-04-28T12:39:04.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.551+0000 7f4e87400640 1 -- 172.21.15.131:0/341810820 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f4e8805a890 con 0x7f4e90160ae0 2024-04-28T12:39:04.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.554+0000 7f4e95e00640 1 -- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4e6c06c8f0 msgr2=0x7f4e6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:04.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.554+0000 7f4e95e00640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4e6c06c8f0 0x7f4e6c06edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4e80003d90 tx=0x7f4e80009040 comp rx=0 tx=0).stop 2024-04-28T12:39:04.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.554+0000 7f4e95e00640 1 -- 172.21.15.131:0/341810820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e90160ae0 msgr2=0x7f4e9015c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:04.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.554+0000 7f4e95e00640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e90160ae0 0x7f4e9015c470 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4e88007e40 tx=0x7f4e8800e4b0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.554+0000 7f4e95e00640 1 -- 172.21.15.131:0/341810820 shutdown_connections 2024-04-28T12:39:04.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.554+0000 7f4e95e00640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4e6c06c8f0 0x7f4e6c06edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.554+0000 7f4e95e00640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e90160ae0 0x7f4e9015c470 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.554+0000 7f4e95e00640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e9013fc90 0x7f4e9015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.554+0000 7f4e95e00640 1 --2- 172.21.15.131:0/341810820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e900d0da0 0x7f4e9006ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.554+0000 7f4e95e00640 1 -- 172.21.15.131:0/341810820 >> 172.21.15.131:0/341810820 conn(0x7f4e900682a0 msgr2=0x7f4e900d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:04.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.554+0000 7f4e95e00640 1 -- 172.21.15.131:0/341810820 shutdown_connections 2024-04-28T12:39:04.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.554+0000 7f4e95e00640 1 -- 172.21.15.131:0/341810820 wait complete. 2024-04-28T12:39:04.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-28T12:39:04.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.767+0000 7f3dd3200640 1 Processor -- start 2024-04-28T12:39:04.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.768+0000 7f3dd3200640 1 -- start start 2024-04-28T12:39:04.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.770+0000 7f3dd3200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3dcc13fc90 0x7f3dcc160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:04.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.770+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dcc15df70 0x7f3dcc142c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:04.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.770+0000 7f3dd3200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dcc15e3b0 con 0x7f3dcc1431d0 2024-04-28T12:39:04.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.770+0000 7f3dd3200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f3dcc161a10 con 0x7f3dcc13fc90 2024-04-28T12:39:04.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.770+0000 7f3dd3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc161170 con 0x7f3dcc15df70 2024-04-28T12:39:04.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.770+0000 7f3dcbe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3dcc13fc90 0x7f3dcc160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:04.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.770+0000 7f3dcbe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3dcc13fc90 0x7f3dcc160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48108/0 (socket says 172.21.15.131:48108) 2024-04-28T12:39:04.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.770+0000 7f3dcbe00640 1 -- 172.21.15.131:0/1806823767 learned_addr learned my addr 172.21.15.131:0/1806823767 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:04.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.770+0000 7f3dcbe00640 1 -- 172.21.15.131:0/1806823767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dcc15df70 msgr2=0x7f3dcc142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:04.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.770+0000 7f3dc3e00640 1 --2- 172.21.15.131:0/1806823767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dcc15df70 0x7f3dcc142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:04.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.771+0000 7f3dcbe00640 1 --2- 172.21.15.131:0/1806823767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dcc15df70 0x7f3dcc142c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.771+0000 7f3dcbe00640 1 -- 172.21.15.131:0/1806823767 >> v1:172.21.15.157:6789/0 conn(0x7f3dcc1431d0 legacy=0x7f3dcc0d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:04.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.771+0000 7f3dcbe00640 1 -- 172.21.15.131:0/1806823767 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc161450 con 0x7f3dcc13fc90 2024-04-28T12:39:04.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.771+0000 7f3dc3e00640 1 --2- 172.21.15.131:0/1806823767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dcc15df70 0x7f3dcc142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:04.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.772+0000 7f3dcbe00640 1 --2- 172.21.15.131:0/1806823767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dcc13fc90 0x7f3dcc160ae0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009870 tx=0x7f3dbc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc080970772cec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:04.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.772+0000 7f3dcb400640 1 -- 172.21.15.131:0/1806823767 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc037070 con 0x7f3dcc13fc90 2024-04-28T12:39:04.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.772+0000 7f3dcb400640 1 -- 172.21.15.131:0/1806823767 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dbc032440 con 0x7f3dcc13fc90 2024-04-28T12:39:04.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.772+0000 7f3dcb400640 1 -- 172.21.15.131:0/1806823767 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc032810 con 0x7f3dcc13fc90 2024-04-28T12:39:04.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.772+0000 7f3dd3200640 1 -- 172.21.15.131:0/1806823767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dcc13fc90 msgr2=0x7f3dcc160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:04.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.772+0000 7f3dd3200640 1 --2- 172.21.15.131:0/1806823767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dcc13fc90 0x7f3dcc160ae0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009870 tx=0x7f3dbc02a210 comp rx=0 tx=0).stop 2024-04-28T12:39:04.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.773+0000 7f3dd3200640 1 -- 172.21.15.131:0/1806823767 shutdown_connections 2024-04-28T12:39:04.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.773+0000 7f3dd3200640 1 --2- 172.21.15.131:0/1806823767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dcc15df70 0x7f3dcc142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.773+0000 7f3dd3200640 1 --2- 172.21.15.131:0/1806823767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dcc13fc90 0x7f3dcc160ae0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.773+0000 7f3dd3200640 1 -- 172.21.15.131:0/1806823767 >> 172.21.15.131:0/1806823767 conn(0x7f3dcc0682a0 msgr2=0x7f3dcc0686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:04.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.773+0000 7f3dd3200640 1 -- 172.21.15.131:0/1806823767 shutdown_connections 2024-04-28T12:39:04.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.773+0000 7f3dd3200640 1 -- 172.21.15.131:0/1806823767 wait complete. 2024-04-28T12:39:04.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.775+0000 7f3dd3200640 1 Processor -- start 2024-04-28T12:39:04.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.775+0000 7f3dd3200640 1 -- start start 2024-04-28T12:39:04.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.776+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dcc13fc90 0x7f3dcc13dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:04.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.776+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dcc1431d0 0x7f3dcc1f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:04.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.776+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dcc15df70 0x7f3dcc15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:04.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.776+0000 7f3dd3200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc140070 con 0x7f3dcc13fc90 2024-04-28T12:39:04.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.776+0000 7f3dd3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc142770 con 0x7f3dcc1431d0 2024-04-28T12:39:04.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.776+0000 7f3dd3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc1401f0 con 0x7f3dcc15df70 2024-04-28T12:39:04.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.776+0000 7f3dcbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dcc13fc90 0x7f3dcc13dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:04.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.776+0000 7f3dcbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dcc13fc90 0x7f3dcc13dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48112/0 (socket says 172.21.15.131:48112) 2024-04-28T12:39:04.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.777+0000 7f3dcbe00640 1 -- 172.21.15.131:0/2319382652 learned_addr learned my addr 172.21.15.131:0/2319382652 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:04.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.777+0000 7f3dcbe00640 1 -- 172.21.15.131:0/2319382652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dcc15df70 msgr2=0x7f3dcc15bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:04.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.777+0000 7f3dcbe00640 1 --2- 172.21.15.131:0/2319382652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dcc15df70 0x7f3dcc15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.777+0000 7f3dcbe00640 1 -- 172.21.15.131:0/2319382652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dcc1431d0 msgr2=0x7f3dcc1f68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:04.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.777+0000 7f3dcbe00640 1 --2- 172.21.15.131:0/2319382652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dcc1431d0 0x7f3dcc1f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:04.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.777+0000 7f3dcbe00640 1 -- 172.21.15.131:0/2319382652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc1f6e00 con 0x7f3dcc13fc90 2024-04-28T12:39:04.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.777+0000 7f3dcbe00640 1 --2- 172.21.15.131:0/2319382652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dcc13fc90 0x7f3dcc13dc80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009870 tx=0x7f3dbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:04.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.777+0000 7f3dc8c00640 1 -- 172.21.15.131:0/2319382652 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc041020 con 0x7f3dcc13fc90 2024-04-28T12:39:04.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.777+0000 7f3dc8c00640 1 -- 172.21.15.131:0/2319382652 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dbc030970 con 0x7f3dcc13fc90 2024-04-28T12:39:04.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.777+0000 7f3dc8c00640 1 -- 172.21.15.131:0/2319382652 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc037040 con 0x7f3dcc13fc90 2024-04-28T12:39:04.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.778+0000 7f3dd3200640 1 -- 172.21.15.131:0/2319382652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc0b3dc0 con 0x7f3dcc13fc90 2024-04-28T12:39:04.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.778+0000 7f3dd3200640 1 -- 172.21.15.131:0/2319382652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc0b4380 con 0x7f3dcc13fc90 2024-04-28T12:39:04.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.779+0000 7f3dd3200640 1 -- 172.21.15.131:0/2319382652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc069350 con 0x7f3dcc13fc90 2024-04-28T12:39:04.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.780+0000 7f3dc8c00640 1 -- 172.21.15.131:0/2319382652 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3dbc044020 con 0x7f3dcc13fc90 2024-04-28T12:39:04.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.780+0000 7f3dc8c00640 1 --2- 172.21.15.131:0/2319382652 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3da806c8f0 0x7f3da806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:04.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.781+0000 7f3dc8c00640 1 -- 172.21.15.131:0/2319382652 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f3dbc0ac420 con 0x7f3dcc13fc90 2024-04-28T12:39:04.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.781+0000 7f3dc3e00640 1 --2- 172.21.15.131:0/2319382652 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3da806c8f0 0x7f3da806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:04.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.782+0000 7f3dc3e00640 1 --2- 172.21.15.131:0/2319382652 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3da806c8f0 0x7f3da806edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3db4006fd0 tx=0x7f3db4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:04.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:04.785+0000 7f3dc8c00640 1 -- 172.21.15.131:0/2319382652 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3dbc04b030 con 0x7f3dcc13fc90 2024-04-28T12:39:05.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.062+0000 7f3dd3200640 1 -- 172.21.15.131:0/2319382652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f3dcc15fac0 con 0x7f3dcc13fc90 2024-04-28T12:39:05.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.067+0000 7f3dc8c00640 1 -- 172.21.15.131:0/2319382652 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f3dbc07a970 con 0x7f3dcc13fc90 2024-04-28T12:39:05.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.070+0000 7f3dd3200640 1 -- 172.21.15.131:0/2319382652 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3da806c8f0 msgr2=0x7f3da806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:05.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.070+0000 7f3dd3200640 1 --2- 172.21.15.131:0/2319382652 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3da806c8f0 0x7f3da806edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3db4006fd0 tx=0x7f3db4008040 comp rx=0 tx=0).stop 2024-04-28T12:39:05.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.070+0000 7f3dd3200640 1 -- 172.21.15.131:0/2319382652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dcc13fc90 msgr2=0x7f3dcc13dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:05.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.070+0000 7f3dd3200640 1 --2- 172.21.15.131:0/2319382652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dcc13fc90 0x7f3dcc13dc80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009870 tx=0x7f3dbc0046c0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.070+0000 7f3dd3200640 1 -- 172.21.15.131:0/2319382652 shutdown_connections 2024-04-28T12:39:05.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.070+0000 7f3dd3200640 1 --2- 172.21.15.131:0/2319382652 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f3da806c8f0 0x7f3da806edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.070+0000 7f3dd3200640 1 --2- 172.21.15.131:0/2319382652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3dcc15df70 0x7f3dcc15bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.070+0000 7f3dd3200640 1 --2- 172.21.15.131:0/2319382652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3dcc1431d0 0x7f3dcc1f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.070+0000 7f3dd3200640 1 --2- 172.21.15.131:0/2319382652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3dcc13fc90 0x7f3dcc13dc80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.070+0000 7f3dd3200640 1 -- 172.21.15.131:0/2319382652 >> 172.21.15.131:0/2319382652 conn(0x7f3dcc0682a0 msgr2=0x7f3dcc15f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:05.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.071+0000 7f3dd3200640 1 -- 172.21.15.131:0/2319382652 shutdown_connections 2024-04-28T12:39:05.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.071+0000 7f3dd3200640 1 -- 172.21.15.131:0/2319382652 wait complete. 2024-04-28T12:39:05.086 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-28T12:39:05.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.279+0000 7fee6b200640 1 Processor -- start 2024-04-28T12:39:05.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.280+0000 7fee6b200640 1 -- start start 2024-04-28T12:39:05.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.281+0000 7fee6b200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fee6c0d2630 0x7fee6c0cff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.281+0000 7fee6b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fee6c0703e0 0x7fee6c06dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.281+0000 7fee6b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6c06e320 0x7fee6c162d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.281+0000 7fee6b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fee6c0d0470 con 0x7fee6c0703e0 2024-04-28T12:39:05.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.281+0000 7fee6b200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fee6c0708c0 con 0x7fee6c0d2630 2024-04-28T12:39:05.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.281+0000 7fee6b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fee6c070ba0 con 0x7fee6c06e320 2024-04-28T12:39:05.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.281+0000 7fee69400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fee6c0703e0 0x7fee6c06dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.281+0000 7fee69400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fee6c0703e0 0x7fee6c06dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48116/0 (socket says 172.21.15.131:48116) 2024-04-28T12:39:05.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.281+0000 7fee69400640 1 -- 172.21.15.131:0/679805069 learned_addr learned my addr 172.21.15.131:0/679805069 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:05.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.282+0000 7fee6a800640 1 --2- 172.21.15.131:0/679805069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6c06e320 0x7fee6c162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.282+0000 7fee69e00640 1 --2- 172.21.15.131:0/679805069 >> v2:172.21.15.157:3300/0 conn(0x7fee6c0d2630 0x7fee6c0cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.282+0000 7fee69400640 1 -- 172.21.15.131:0/679805069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6c06e320 msgr2=0x7fee6c162d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:05.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.282+0000 7fee69400640 1 --2- 172.21.15.131:0/679805069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6c06e320 0x7fee6c162d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.282+0000 7fee69400640 1 -- 172.21.15.131:0/679805069 >> v2:172.21.15.157:3300/0 conn(0x7fee6c0d2630 msgr2=0x7fee6c0cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.282+0000 7fee69400640 1 --2- 172.21.15.131:0/679805069 >> v2:172.21.15.157:3300/0 conn(0x7fee6c0d2630 0x7fee6c0cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.282+0000 7fee69400640 1 -- 172.21.15.131:0/679805069 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6c070e80 con 0x7fee6c0703e0 2024-04-28T12:39:05.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.283+0000 7fee69400640 1 --2- 172.21.15.131:0/679805069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6c0703e0 0x7fee6c06dde0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fee6c073100 tx=0x7fee5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2cc993a41b6e93 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.283+0000 7fee68a00640 1 -- 172.21.15.131:0/679805069 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58037070 con 0x7fee6c0703e0 2024-04-28T12:39:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.284+0000 7fee68a00640 1 -- 172.21.15.131:0/679805069 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee58032440 con 0x7fee6c0703e0 2024-04-28T12:39:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.284+0000 7fee68a00640 1 -- 172.21.15.131:0/679805069 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58032810 con 0x7fee6c0703e0 2024-04-28T12:39:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.284+0000 7fee6b200640 1 -- 172.21.15.131:0/679805069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6c0703e0 msgr2=0x7fee6c06dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.284+0000 7fee6b200640 1 --2- 172.21.15.131:0/679805069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6c0703e0 0x7fee6c06dde0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fee6c073100 tx=0x7fee5802a1e0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.284+0000 7fee6b200640 1 -- 172.21.15.131:0/679805069 shutdown_connections 2024-04-28T12:39:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.284+0000 7fee6b200640 1 --2- 172.21.15.131:0/679805069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6c0703e0 0x7fee6c06dde0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.284+0000 7fee6b200640 1 --2- 172.21.15.131:0/679805069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6c06e320 0x7fee6c162d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.284+0000 7fee6b200640 1 --2- 172.21.15.131:0/679805069 >> v2:172.21.15.157:3300/0 conn(0x7fee6c0d2630 0x7fee6c0cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.284+0000 7fee6b200640 1 -- 172.21.15.131:0/679805069 >> 172.21.15.131:0/679805069 conn(0x7fee6c0d2b10 msgr2=0x7fee6c0d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.284+0000 7fee6b200640 1 -- 172.21.15.131:0/679805069 shutdown_connections 2024-04-28T12:39:05.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.284+0000 7fee6b200640 1 -- 172.21.15.131:0/679805069 wait complete. 2024-04-28T12:39:05.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.286+0000 7fee6b200640 1 Processor -- start 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.286+0000 7fee6b200640 1 -- start start 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.287+0000 7fee6b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee6c06e320 0x7fee6c156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.287+0000 7fee6b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6c0703e0 0x7fee6c0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.287+0000 7fee6b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6c0d2630 0x7fee6c0c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.287+0000 7fee6b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c0d38c0 con 0x7fee6c0d2630 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.287+0000 7fee6b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c071440 con 0x7fee6c06e320 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.287+0000 7fee6b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fee6c1632b0 con 0x7fee6c0703e0 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.287+0000 7fee6a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6c0d2630 0x7fee6c0c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.287+0000 7fee6a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6c0d2630 0x7fee6c0c87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48124/0 (socket says 172.21.15.131:48124) 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.287+0000 7fee6a800640 1 -- 172.21.15.131:0/3035543702 learned_addr learned my addr 172.21.15.131:0/3035543702 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee69e00640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee6c06e320 0x7fee6c156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee69400640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6c0703e0 0x7fee6c0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee6a800640 1 -- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6c0703e0 msgr2=0x7fee6c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee6a800640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6c0703e0 0x7fee6c0c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee6a800640 1 -- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee6c06e320 msgr2=0x7fee6c156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee6a800640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee6c06e320 0x7fee6c156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee6a800640 1 -- 172.21.15.131:0/3035543702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6c0c7020 con 0x7fee6c0d2630 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee6a800640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6c0d2630 0x7fee6c0c87e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fee64007e60 tx=0x7fee6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee62000640 1 -- 172.21.15.131:0/3035543702 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64014620 con 0x7fee6c0d2630 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee62000640 1 -- 172.21.15.131:0/3035543702 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee64019070 con 0x7fee6c0d2630 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee6b200640 1 -- 172.21.15.131:0/3035543702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6c0c7300 con 0x7fee6c0d2630 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee62000640 1 -- 172.21.15.131:0/3035543702 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64014a20 con 0x7fee6c0d2630 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.288+0000 7fee6b200640 1 -- 172.21.15.131:0/3035543702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6c156b70 con 0x7fee6c0d2630 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.290+0000 7fee6b200640 1 -- 172.21.15.131:0/3035543702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee6c069350 con 0x7fee6c0d2630 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.290+0000 7fee62000640 1 -- 172.21.15.131:0/3035543702 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee64014bc0 con 0x7fee6c0d2630 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.291+0000 7fee62000640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fee4c06c8f0 0x7fee4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.291+0000 7fee62000640 1 -- 172.21.15.131:0/3035543702 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fee6408c240 con 0x7fee6c0d2630 2024-04-28T12:39:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.292+0000 7fee69e00640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fee4c06c8f0 0x7fee4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.293+0000 7fee69e00640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fee4c06c8f0 0x7fee4c06edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fee5c006fd0 tx=0x7fee5c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:05.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.296+0000 7fee62000640 1 -- 172.21.15.131:0/3035543702 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee6405a820 con 0x7fee6c0d2630 2024-04-28T12:39:05.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.547+0000 7fee6b200640 1 -- 172.21.15.131:0/3035543702 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fee6c074010 con 0x7fee6c0d2630 2024-04-28T12:39:05.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.551+0000 7fee62000640 1 -- 172.21.15.131:0/3035543702 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fee6405a120 con 0x7fee6c0d2630 2024-04-28T12:39:05.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.555+0000 7fee6b200640 1 -- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fee4c06c8f0 msgr2=0x7fee4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:05.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.555+0000 7fee6b200640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fee4c06c8f0 0x7fee4c06edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fee5c006fd0 tx=0x7fee5c008040 comp rx=0 tx=0).stop 2024-04-28T12:39:05.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.555+0000 7fee6b200640 1 -- 172.21.15.131:0/3035543702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6c0d2630 msgr2=0x7fee6c0c87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:05.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.555+0000 7fee6b200640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6c0d2630 0x7fee6c0c87e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fee64007e60 tx=0x7fee6400e4b0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.555+0000 7fee6b200640 1 -- 172.21.15.131:0/3035543702 shutdown_connections 2024-04-28T12:39:05.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.555+0000 7fee6b200640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fee4c06c8f0 0x7fee4c06edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.555+0000 7fee6b200640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6c0d2630 0x7fee6c0c87e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.555+0000 7fee6b200640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee6c0703e0 0x7fee6c0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.555+0000 7fee6b200640 1 --2- 172.21.15.131:0/3035543702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee6c06e320 0x7fee6c156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.555+0000 7fee6b200640 1 -- 172.21.15.131:0/3035543702 >> 172.21.15.131:0/3035543702 conn(0x7fee6c0d2b10 msgr2=0x7fee6c06d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:05.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.555+0000 7fee6b200640 1 -- 172.21.15.131:0/3035543702 shutdown_connections 2024-04-28T12:39:05.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.555+0000 7fee6b200640 1 -- 172.21.15.131:0/3035543702 wait complete. 2024-04-28T12:39:05.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-28T12:39:05.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.769+0000 7f7a50a00640 1 Processor -- start 2024-04-28T12:39:05.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.770+0000 7f7a50a00640 1 -- start start 2024-04-28T12:39:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.771+0000 7f7a50a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7a4c15fcd0 0x7f7a4c0b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.771+0000 7f7a50a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c1600b0 0x7f7a4c0b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.771+0000 7f7a50a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7a4c072500 0x7f7a4c072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.771+0000 7f7a50a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7a4c0b4b20 con 0x7f7a4c072500 2024-04-28T12:39:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.771+0000 7f7a50a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7a4c0b3c90 con 0x7f7a4c15fcd0 2024-04-28T12:39:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.771+0000 7f7a50a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4c072e90 con 0x7f7a4c1600b0 2024-04-28T12:39:05.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.772+0000 7f7a4b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7a4c072500 0x7f7a4c072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.772+0000 7f7a4b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7a4c072500 0x7f7a4c072950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48134/0 (socket says 172.21.15.131:48134) 2024-04-28T12:39:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.772+0000 7f7a4b200640 1 -- 172.21.15.131:0/3275840790 learned_addr learned my addr 172.21.15.131:0/3275840790 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.772+0000 7f7a4a800640 1 --2- 172.21.15.131:0/3275840790 >> v2:172.21.15.157:3300/0 conn(0x7f7a4c15fcd0 0x7f7a4c0b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.772+0000 7f7a41e00640 1 --2- 172.21.15.131:0/3275840790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c1600b0 0x7f7a4c0b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.772+0000 7f7a4b200640 1 -- 172.21.15.131:0/3275840790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c1600b0 msgr2=0x7f7a4c0b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.772+0000 7f7a4b200640 1 --2- 172.21.15.131:0/3275840790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c1600b0 0x7f7a4c0b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.772+0000 7f7a4b200640 1 -- 172.21.15.131:0/3275840790 >> v2:172.21.15.157:3300/0 conn(0x7f7a4c15fcd0 msgr2=0x7f7a4c0b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.772+0000 7f7a4b200640 1 --2- 172.21.15.131:0/3275840790 >> v2:172.21.15.157:3300/0 conn(0x7f7a4c15fcd0 0x7f7a4c0b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.772+0000 7f7a4b200640 1 -- 172.21.15.131:0/3275840790 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c0b3f10 con 0x7f7a4c072500 2024-04-28T12:39:05.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.773+0000 7f7a4b200640 1 --2- 172.21.15.131:0/3275840790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a4c072500 0x7f7a4c072950 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0b32c0 tx=0x7f7a4402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a74c2638539ca1c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:05.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.774+0000 7f7a49e00640 1 -- 172.21.15.131:0/3275840790 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44036070 con 0x7f7a4c072500 2024-04-28T12:39:05.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.774+0000 7f7a49e00640 1 -- 172.21.15.131:0/3275840790 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a44031600 con 0x7f7a4c072500 2024-04-28T12:39:05.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.774+0000 7f7a49e00640 1 -- 172.21.15.131:0/3275840790 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a440319d0 con 0x7f7a4c072500 2024-04-28T12:39:05.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.774+0000 7f7a50a00640 1 -- 172.21.15.131:0/3275840790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a4c072500 msgr2=0x7f7a4c072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:05.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.774+0000 7f7a50a00640 1 --2- 172.21.15.131:0/3275840790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a4c072500 0x7f7a4c072950 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0b32c0 tx=0x7f7a4402a6a0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.775+0000 7f7a50a00640 1 -- 172.21.15.131:0/3275840790 shutdown_connections 2024-04-28T12:39:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.775+0000 7f7a50a00640 1 --2- 172.21.15.131:0/3275840790 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a4c072500 0x7f7a4c072950 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.775+0000 7f7a50a00640 1 --2- 172.21.15.131:0/3275840790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c1600b0 0x7f7a4c0b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.775+0000 7f7a50a00640 1 --2- 172.21.15.131:0/3275840790 >> v2:172.21.15.157:3300/0 conn(0x7f7a4c15fcd0 0x7f7a4c0b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.775+0000 7f7a50a00640 1 -- 172.21.15.131:0/3275840790 >> 172.21.15.131:0/3275840790 conn(0x7f7a4c0b26b0 msgr2=0x7f7a4c0b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.775+0000 7f7a50a00640 1 -- 172.21.15.131:0/3275840790 shutdown_connections 2024-04-28T12:39:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.775+0000 7f7a50a00640 1 -- 172.21.15.131:0/3275840790 wait complete. 2024-04-28T12:39:05.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.776+0000 7f7a50a00640 1 Processor -- start 2024-04-28T12:39:05.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.777+0000 7f7a50a00640 1 -- start start 2024-04-28T12:39:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.777+0000 7f7a50a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c072500 0x7f7a4c156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.777+0000 7f7a50a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a4c15fcd0 0x7f7a4c1f8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a50a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c1600b0 0x7f7a4c148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a50a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c0b47b0 con 0x7f7a4c15fcd0 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a50a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c0b44d0 con 0x7f7a4c1600b0 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a50a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4c0b4fb0 con 0x7f7a4c072500 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a41e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a4c15fcd0 0x7f7a4c1f8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a41e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a4c15fcd0 0x7f7a4c1f8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48142/0 (socket says 172.21.15.131:48142) 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a41e00640 1 -- 172.21.15.131:0/688840205 learned_addr learned my addr 172.21.15.131:0/688840205 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a4a800640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c072500 0x7f7a4c156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a4b200640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c1600b0 0x7f7a4c148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a41e00640 1 -- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c072500 msgr2=0x7f7a4c156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a41e00640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c072500 0x7f7a4c156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a41e00640 1 -- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c1600b0 msgr2=0x7f7a4c148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a41e00640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c1600b0 0x7f7a4c148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a41e00640 1 -- 172.21.15.131:0/688840205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c155ba0 con 0x7f7a4c15fcd0 2024-04-28T12:39:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a4a800640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c072500 0x7f7a4c156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.778+0000 7f7a4b200640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c1600b0 0x7f7a4c148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.779+0000 7f7a41e00640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a4c15fcd0 0x7f7a4c1f8ca0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400d800 tx=0x7f7a3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.779+0000 7f7a43400640 1 -- 172.21.15.131:0/688840205 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400b6c0 con 0x7f7a4c15fcd0 2024-04-28T12:39:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.779+0000 7f7a43400640 1 -- 172.21.15.131:0/688840205 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a34019070 con 0x7f7a4c15fcd0 2024-04-28T12:39:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.779+0000 7f7a50a00640 1 -- 172.21.15.131:0/688840205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4c155e80 con 0x7f7a4c15fcd0 2024-04-28T12:39:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.779+0000 7f7a43400640 1 -- 172.21.15.131:0/688840205 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400ba80 con 0x7f7a4c15fcd0 2024-04-28T12:39:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.779+0000 7f7a50a00640 1 -- 172.21.15.131:0/688840205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4c156440 con 0x7f7a4c15fcd0 2024-04-28T12:39:05.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.780+0000 7f7a50a00640 1 -- 172.21.15.131:0/688840205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a4c069350 con 0x7f7a4c15fcd0 2024-04-28T12:39:05.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.781+0000 7f7a43400640 1 -- 172.21.15.131:0/688840205 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a3400bc20 con 0x7f7a4c15fcd0 2024-04-28T12:39:05.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.782+0000 7f7a43400640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7a2806c8f0 0x7f7a2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:05.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.782+0000 7f7a43400640 1 -- 172.21.15.131:0/688840205 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f7a3408be30 con 0x7f7a4c15fcd0 2024-04-28T12:39:05.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.783+0000 7f7a4a800640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7a2806c8f0 0x7f7a2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:05.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.784+0000 7f7a4a800640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7a2806c8f0 0x7f7a2806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c006fd0 tx=0x7f7a3c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:05.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:05.786+0000 7f7a43400640 1 -- 172.21.15.131:0/688840205 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7a3405a420 con 0x7f7a4c15fcd0 2024-04-28T12:39:06.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.067+0000 7f7a50a00640 1 -- 172.21.15.131:0/688840205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f7a4c06e910 con 0x7f7a4c15fcd0 2024-04-28T12:39:06.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.072+0000 7f7a43400640 1 -- 172.21.15.131:0/688840205 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v10) ==== 97+0+0 (secure 0 0 0) 0x7f7a34059d20 con 0x7f7a4c15fcd0 2024-04-28T12:39:06.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.074+0000 7f7a50a00640 1 -- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7a2806c8f0 msgr2=0x7f7a2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.074+0000 7f7a50a00640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7a2806c8f0 0x7f7a2806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c006fd0 tx=0x7f7a3c008040 comp rx=0 tx=0).stop 2024-04-28T12:39:06.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.074+0000 7f7a50a00640 1 -- 172.21.15.131:0/688840205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a4c15fcd0 msgr2=0x7f7a4c1f8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.074+0000 7f7a50a00640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a4c15fcd0 0x7f7a4c1f8ca0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400d800 tx=0x7f7a3400dcc0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.074+0000 7f7a50a00640 1 -- 172.21.15.131:0/688840205 shutdown_connections 2024-04-28T12:39:06.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.074+0000 7f7a50a00640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7a2806c8f0 0x7f7a2806edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.074+0000 7f7a50a00640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a4c1600b0 0x7f7a4c148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.074+0000 7f7a50a00640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a4c15fcd0 0x7f7a4c1f8ca0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.074+0000 7f7a50a00640 1 --2- 172.21.15.131:0/688840205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7a4c072500 0x7f7a4c156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.074+0000 7f7a50a00640 1 -- 172.21.15.131:0/688840205 >> 172.21.15.131:0/688840205 conn(0x7f7a4c0b26b0 msgr2=0x7f7a4c15ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:06.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.075+0000 7f7a50a00640 1 -- 172.21.15.131:0/688840205 shutdown_connections 2024-04-28T12:39:06.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.075+0000 7f7a50a00640 1 -- 172.21.15.131:0/688840205 wait complete. 2024-04-28T12:39:06.090 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-28T12:39:06.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.282+0000 7f7885e00640 1 Processor -- start 2024-04-28T12:39:06.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.283+0000 7f7885e00640 1 -- start start 2024-04-28T12:39:06.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.284+0000 7f7885e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f78800b5c10 0x7f7880072500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.284+0000 7f7885e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f78801600b0 0x7f7880072a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.284+0000 7f7885e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78800b3c90 con 0x7f788015fcd0 2024-04-28T12:39:06.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.284+0000 7f7885e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f78800b4b20 con 0x7f78801600b0 2024-04-28T12:39:06.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.284+0000 7f7885e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f78800af8d0 con 0x7f78800b5c10 2024-04-28T12:39:06.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.284+0000 7f7885400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f78801600b0 0x7f7880072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:06.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.284+0000 7f7884a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f788015fcd0 0x7f78800b5500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38138/0 (socket says 172.21.15.131:38138) 2024-04-28T12:39:06.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.284+0000 7f7884a00640 1 -- 172.21.15.131:0/2099740910 learned_addr learned my addr 172.21.15.131:0/2099740910 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:06.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.284+0000 7f7877e00640 1 --2- 172.21.15.131:0/2099740910 >> v2:172.21.15.157:3300/0 conn(0x7f78800b5c10 0x7f7880072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:06.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.284+0000 7f7885400640 1 -- 172.21.15.131:0/2099740910 >> v2:172.21.15.157:3300/0 conn(0x7f78800b5c10 msgr2=0x7f7880072500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.284+0000 7f7885400640 1 --2- 172.21.15.131:0/2099740910 >> v2:172.21.15.157:3300/0 conn(0x7f78800b5c10 0x7f7880072500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.284+0000 7f7885400640 1 -- 172.21.15.131:0/2099740910 >> v1:172.21.15.131:6789/0 conn(0x7f788015fcd0 legacy=0x7f78800b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.285+0000 7f7885400640 1 -- 172.21.15.131:0/2099740910 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78800b3f10 con 0x7f78801600b0 2024-04-28T12:39:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.285+0000 7f787fe00640 1 -- 172.21.15.131:0/2099740910 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 911715455 0 0) 0x7f7870001110 con 0x7f788015fcd0 2024-04-28T12:39:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.286+0000 7f7885400640 1 --2- 172.21.15.131:0/2099740910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78801600b0 0x7f7880072a40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f78800700f0 tx=0x7f787802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82db951c30fa3f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.286+0000 7f787fe00640 1 -- 172.21.15.131:0/2099740910 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878007740 con 0x7f78801600b0 2024-04-28T12:39:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.286+0000 7f787fe00640 1 -- 172.21.15.131:0/2099740910 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78780078e0 con 0x7f78801600b0 2024-04-28T12:39:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.286+0000 7f787fe00640 1 -- 172.21.15.131:0/2099740910 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878007cb0 con 0x7f78801600b0 2024-04-28T12:39:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.286+0000 7f7885e00640 1 -- 172.21.15.131:0/2099740910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78801600b0 msgr2=0x7f7880072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.286+0000 7f7885e00640 1 --2- 172.21.15.131:0/2099740910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78801600b0 0x7f7880072a40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f78800700f0 tx=0x7f787802a680 comp rx=0 tx=0).stop 2024-04-28T12:39:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.286+0000 7f7885e00640 1 -- 172.21.15.131:0/2099740910 shutdown_connections 2024-04-28T12:39:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.286+0000 7f7885e00640 1 --2- 172.21.15.131:0/2099740910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78801600b0 0x7f7880072a40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.286+0000 7f7885e00640 1 --2- 172.21.15.131:0/2099740910 >> v2:172.21.15.157:3300/0 conn(0x7f78800b5c10 0x7f7880072500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.286+0000 7f7885e00640 1 -- 172.21.15.131:0/2099740910 >> 172.21.15.131:0/2099740910 conn(0x7f78800b26b0 msgr2=0x7f78800b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:06.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.286+0000 7f7885e00640 1 -- 172.21.15.131:0/2099740910 shutdown_connections 2024-04-28T12:39:06.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.287+0000 7f7885e00640 1 -- 172.21.15.131:0/2099740910 wait complete. 2024-04-28T12:39:06.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.288+0000 7f7885e00640 1 Processor -- start 2024-04-28T12:39:06.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.288+0000 7f7885e00640 1 -- start start 2024-04-28T12:39:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.289+0000 7f7885e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78800b5c10 0x7f7880148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.289+0000 7f7885e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f788015fcd0 0x7f78801f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.289+0000 7f7885e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78801600b0 0x7f78800b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.289+0000 7f7885e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f78800b47b0 con 0x7f78800b5c10 2024-04-28T12:39:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.289+0000 7f7885e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78800b44d0 con 0x7f78801600b0 2024-04-28T12:39:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.289+0000 7f7885e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f78800b4fb0 con 0x7f788015fcd0 2024-04-28T12:39:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.289+0000 7f7884a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78800b5c10 0x7f7880148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.289+0000 7f7884a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78800b5c10 0x7f7880148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48154/0 (socket says 172.21.15.131:48154) 2024-04-28T12:39:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.289+0000 7f7884a00640 1 -- 172.21.15.131:0/677384250 learned_addr learned my addr 172.21.15.131:0/677384250 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.290+0000 7f7884a00640 1 -- 172.21.15.131:0/677384250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f788015fcd0 msgr2=0x7f78801f8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.290+0000 7f7884a00640 1 --2- 172.21.15.131:0/677384250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f788015fcd0 0x7f78801f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.290+0000 7f7884a00640 1 -- 172.21.15.131:0/677384250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78801600b0 msgr2=0x7f78800b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.290+0000 7f7884a00640 1 --2- 172.21.15.131:0/677384250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78801600b0 0x7f78800b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.290+0000 7f7884a00640 1 -- 172.21.15.131:0/677384250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788015bdf0 con 0x7f78800b5c10 2024-04-28T12:39:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.290+0000 7f7884a00640 1 --2- 172.21.15.131:0/677384250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78800b5c10 0x7f7880148950 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f787000c780 tx=0x7f787000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.290+0000 7f787d600640 1 -- 172.21.15.131:0/677384250 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870018070 con 0x7f78800b5c10 2024-04-28T12:39:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.290+0000 7f787d600640 1 -- 172.21.15.131:0/677384250 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78700072d0 con 0x7f78800b5c10 2024-04-28T12:39:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.290+0000 7f7885e00640 1 -- 172.21.15.131:0/677384250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78801f92b0 con 0x7f78800b5c10 2024-04-28T12:39:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.290+0000 7f787d600640 1 -- 172.21.15.131:0/677384250 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78700047e0 con 0x7f78800b5c10 2024-04-28T12:39:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.291+0000 7f7885e00640 1 -- 172.21.15.131:0/677384250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7880147e70 con 0x7f78800b5c10 2024-04-28T12:39:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.292+0000 7f7885e00640 1 -- 172.21.15.131:0/677384250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7880069350 con 0x7f78800b5c10 2024-04-28T12:39:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.293+0000 7f787d600640 1 -- 172.21.15.131:0/677384250 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f787001b070 con 0x7f78800b5c10 2024-04-28T12:39:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.294+0000 7f787d600640 1 --2- 172.21.15.131:0/677384250 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f785c06c8f0 0x7f785c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.294+0000 7f787d600640 1 -- 172.21.15.131:0/677384250 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f787008acc0 con 0x7f78800b5c10 2024-04-28T12:39:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.295+0000 7f7877e00640 1 --2- 172.21.15.131:0/677384250 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f785c06c8f0 0x7f785c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.296+0000 7f7877e00640 1 --2- 172.21.15.131:0/677384250 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f785c06c8f0 0x7f785c06edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7868006fd0 tx=0x7f7868008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.298+0000 7f787d600640 1 -- 172.21.15.131:0/677384250 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7870091080 con 0x7f78800b5c10 2024-04-28T12:39:06.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.554+0000 7f7885e00640 1 -- 172.21.15.131:0/677384250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f788015e040 con 0x7f78800b5c10 2024-04-28T12:39:06.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.558+0000 7f787d600640 1 -- 172.21.15.131:0/677384250 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7f78800b3c90 con 0x7f78800b5c10 2024-04-28T12:39:06.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.561+0000 7f7885e00640 1 -- 172.21.15.131:0/677384250 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f785c06c8f0 msgr2=0x7f785c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.561+0000 7f7885e00640 1 --2- 172.21.15.131:0/677384250 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f785c06c8f0 0x7f785c06edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7868006fd0 tx=0x7f7868008040 comp rx=0 tx=0).stop 2024-04-28T12:39:06.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.561+0000 7f7885e00640 1 -- 172.21.15.131:0/677384250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78800b5c10 msgr2=0x7f7880148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.561+0000 7f7885e00640 1 --2- 172.21.15.131:0/677384250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78800b5c10 0x7f7880148950 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f787000c780 tx=0x7f787000cc40 comp rx=0 tx=0).stop 2024-04-28T12:39:06.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.562+0000 7f7885e00640 1 -- 172.21.15.131:0/677384250 shutdown_connections 2024-04-28T12:39:06.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.562+0000 7f7885e00640 1 --2- 172.21.15.131:0/677384250 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f785c06c8f0 0x7f785c06edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.562+0000 7f7885e00640 1 --2- 172.21.15.131:0/677384250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78801600b0 0x7f78800b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.562+0000 7f7885e00640 1 --2- 172.21.15.131:0/677384250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f788015fcd0 0x7f78801f8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.562+0000 7f7885e00640 1 --2- 172.21.15.131:0/677384250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78800b5c10 0x7f7880148950 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.562+0000 7f7885e00640 1 -- 172.21.15.131:0/677384250 >> 172.21.15.131:0/677384250 conn(0x7f78800b26b0 msgr2=0x7f78800b5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:06.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.562+0000 7f7885e00640 1 -- 172.21.15.131:0/677384250 shutdown_connections 2024-04-28T12:39:06.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.562+0000 7f7885e00640 1 -- 172.21.15.131:0/677384250 wait complete. 2024-04-28T12:39:06.577 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:39:06.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.776+0000 7f119d200640 1 Processor -- start 2024-04-28T12:39:06.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.776+0000 7f119d200640 1 -- start start 2024-04-28T12:39:06.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.777+0000 7f119d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11980cd180 0x7f1198072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.777+0000 7f119d200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f119815dc50 0x7f11980cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f119d200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f11980ce2b0 0x7f11981607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f119d200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f11980cd560 con 0x7f119815dc50 2024-04-28T12:39:06.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f119d200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1198072d00 con 0x7f11980ce2b0 2024-04-28T12:39:06.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f119d200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1198072fe0 con 0x7f11980cd180 2024-04-28T12:39:06.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f118ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f119815dc50 0x7f11980cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:06.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f118ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f119815dc50 0x7f11980cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48160/0 (socket says 172.21.15.131:48160) 2024-04-28T12:39:06.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f118ea00640 1 -- 172.21.15.131:0/2813846887 learned_addr learned my addr 172.21.15.131:0/2813846887 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:06.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f1197400640 1 --2- 172.21.15.131:0/2813846887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11980cd180 0x7f1198072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:06.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f1197e00640 1 --2- 172.21.15.131:0/2813846887 >> v2:172.21.15.157:3300/0 conn(0x7f11980ce2b0 0x7f11981607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:06.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f118ea00640 1 -- 172.21.15.131:0/2813846887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11980cd180 msgr2=0x7f1198072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:06.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f118ea00640 1 --2- 172.21.15.131:0/2813846887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11980cd180 0x7f1198072620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f118ea00640 1 -- 172.21.15.131:0/2813846887 >> v2:172.21.15.157:3300/0 conn(0x7f11980ce2b0 msgr2=0x7f11981607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:06.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f118ea00640 1 --2- 172.21.15.131:0/2813846887 >> v2:172.21.15.157:3300/0 conn(0x7f11980ce2b0 0x7f11981607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.778+0000 7f118ea00640 1 -- 172.21.15.131:0/2813846887 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11980732c0 con 0x7f119815dc50 2024-04-28T12:39:06.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.779+0000 7f118ea00640 1 --2- 172.21.15.131:0/2813846887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f119815dc50 0x7f11980cdd70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1198072080 tx=0x7f118002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c6cbcafdc17638 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:06.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.780+0000 7f1196a00640 1 -- 172.21.15.131:0/2813846887 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180037070 con 0x7f119815dc50 2024-04-28T12:39:06.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.780+0000 7f1196a00640 1 -- 172.21.15.131:0/2813846887 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1180032440 con 0x7f119815dc50 2024-04-28T12:39:06.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.780+0000 7f1196a00640 1 -- 172.21.15.131:0/2813846887 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180032810 con 0x7f119815dc50 2024-04-28T12:39:06.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.780+0000 7f119d200640 1 -- 172.21.15.131:0/2813846887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f119815dc50 msgr2=0x7f11980cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.780+0000 7f119d200640 1 --2- 172.21.15.131:0/2813846887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f119815dc50 0x7f11980cdd70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1198072080 tx=0x7f118002a1e0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.780+0000 7f119d200640 1 -- 172.21.15.131:0/2813846887 shutdown_connections 2024-04-28T12:39:06.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.780+0000 7f119d200640 1 --2- 172.21.15.131:0/2813846887 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f119815dc50 0x7f11980cdd70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.780+0000 7f119d200640 1 --2- 172.21.15.131:0/2813846887 >> v2:172.21.15.157:3300/0 conn(0x7f11980ce2b0 0x7f11981607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.780+0000 7f119d200640 1 --2- 172.21.15.131:0/2813846887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11980cd180 0x7f1198072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.780+0000 7f119d200640 1 -- 172.21.15.131:0/2813846887 >> 172.21.15.131:0/2813846887 conn(0x7f1198068090 msgr2=0x7f1198068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:06.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.780+0000 7f119d200640 1 -- 172.21.15.131:0/2813846887 shutdown_connections 2024-04-28T12:39:06.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.780+0000 7f119d200640 1 -- 172.21.15.131:0/2813846887 wait complete. 2024-04-28T12:39:06.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.782+0000 7f119d200640 1 Processor -- start 2024-04-28T12:39:06.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.782+0000 7f119d200640 1 -- start start 2024-04-28T12:39:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.783+0000 7f119d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11980cd180 0x7f1198160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.783+0000 7f119d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11980ce2b0 0x7f1198145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.783+0000 7f119d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f119815dc50 0x7f11980aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.783+0000 7f119d200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f119815e0d0 con 0x7f119815dc50 2024-04-28T12:39:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.783+0000 7f119d200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1198073da0 con 0x7f11980ce2b0 2024-04-28T12:39:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.783+0000 7f119d200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1198160d10 con 0x7f11980cd180 2024-04-28T12:39:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.783+0000 7f1197e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f119815dc50 0x7f11980aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.783+0000 7f1197e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f119815dc50 0x7f11980aa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48172/0 (socket says 172.21.15.131:48172) 2024-04-28T12:39:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.783+0000 7f1197e00640 1 -- 172.21.15.131:0/746181229 learned_addr learned my addr 172.21.15.131:0/746181229 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f1197400640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11980cd180 0x7f1198160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f1197e00640 1 -- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11980cd180 msgr2=0x7f1198160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f118ea00640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11980ce2b0 0x7f1198145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:06.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f1197e00640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11980cd180 0x7f1198160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f1197e00640 1 -- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11980ce2b0 msgr2=0x7f1198145d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f1197e00640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11980ce2b0 0x7f1198145d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f1197e00640 1 -- 172.21.15.131:0/746181229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1198144040 con 0x7f119815dc50 2024-04-28T12:39:06.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f1197400640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11980cd180 0x7f1198160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:06.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f1197e00640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f119815dc50 0x7f11980aa4b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f119000c7f0 tx=0x7f119000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:06.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f118fe00640 1 -- 172.21.15.131:0/746181229 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190015070 con 0x7f119815dc50 2024-04-28T12:39:06.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f118fe00640 1 -- 172.21.15.131:0/746181229 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1190004220 con 0x7f119815dc50 2024-04-28T12:39:06.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f118fe00640 1 -- 172.21.15.131:0/746181229 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190007620 con 0x7f119815dc50 2024-04-28T12:39:06.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f119d200640 1 -- 172.21.15.131:0/746181229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1198144320 con 0x7f119815dc50 2024-04-28T12:39:06.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.784+0000 7f119d200640 1 -- 172.21.15.131:0/746181229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11981448e0 con 0x7f119815dc50 2024-04-28T12:39:06.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.786+0000 7f119d200640 1 -- 172.21.15.131:0/746181229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1198069350 con 0x7f119815dc50 2024-04-28T12:39:06.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.786+0000 7f118fe00640 1 -- 172.21.15.131:0/746181229 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11900079d0 con 0x7f119815dc50 2024-04-28T12:39:06.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.788+0000 7f118fe00640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f117406c8f0 0x7f117406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:06.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.788+0000 7f118fe00640 1 -- 172.21.15.131:0/746181229 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f119008baf0 con 0x7f119815dc50 2024-04-28T12:39:06.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.789+0000 7f1197400640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f117406c8f0 0x7f117406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:06.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.790+0000 7f1197400640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f117406c8f0 0x7f117406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1188008f10 tx=0x7f1188008460 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:06.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.792+0000 7f118fe00640 1 -- 172.21.15.131:0/746181229 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f119005a040 con 0x7f119815dc50 2024-04-28T12:39:06.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.976+0000 7f119d200640 1 -- 172.21.15.131:0/746181229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11980b5560 con 0x7f119815dc50 2024-04-28T12:39:06.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.978+0000 7f118fe00640 1 -- 172.21.15.131:0/746181229 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13188 (secure 0 0 0) 0x7f1190059940 con 0x7f119815dc50 2024-04-28T12:39:06.975 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:06.975 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":34,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:39:03.035350+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:39:06.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.981+0000 7f119d200640 1 -- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f117406c8f0 msgr2=0x7f117406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.981+0000 7f119d200640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f117406c8f0 0x7f117406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1188008f10 tx=0x7f1188008460 comp rx=0 tx=0).stop 2024-04-28T12:39:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.981+0000 7f119d200640 1 -- 172.21.15.131:0/746181229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f119815dc50 msgr2=0x7f11980aa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.981+0000 7f119d200640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f119815dc50 0x7f11980aa4b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f119000c7f0 tx=0x7f119000ccb0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.981+0000 7f119d200640 1 -- 172.21.15.131:0/746181229 shutdown_connections 2024-04-28T12:39:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.981+0000 7f119d200640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f117406c8f0 0x7f117406edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.981+0000 7f119d200640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f119815dc50 0x7f11980aa4b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.981+0000 7f119d200640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11980ce2b0 0x7f1198145d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.981+0000 7f119d200640 1 --2- 172.21.15.131:0/746181229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11980cd180 0x7f1198160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.981+0000 7f119d200640 1 -- 172.21.15.131:0/746181229 >> 172.21.15.131:0/746181229 conn(0x7f1198068090 msgr2=0x7f11980cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.981+0000 7f119d200640 1 -- 172.21.15.131:0/746181229 shutdown_connections 2024-04-28T12:39:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:06.981+0000 7f119d200640 1 -- 172.21.15.131:0/746181229 wait complete. 2024-04-28T12:39:06.997 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T12:39:06.997 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:39:07.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.178+0000 7f33ede00640 1 Processor -- start 2024-04-28T12:39:07.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.179+0000 7f33ede00640 1 -- start start 2024-04-28T12:39:07.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.180+0000 7f33ede00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f33e806f3a0 0x7f33e80b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.180+0000 7f33ede00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f33e815bd60 0x7f33e80b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.180+0000 7f33ede00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33e815c2a0 con 0x7f33e815b980 2024-04-28T12:39:07.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.180+0000 7f33ede00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f33e815d060 con 0x7f33e806f3a0 2024-04-28T12:39:07.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.180+0000 7f33ede00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f33e80b5090 con 0x7f33e815bd60 2024-04-28T12:39:07.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.180+0000 7f33e7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f33e806f3a0 0x7f33e80b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:07.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.181+0000 7f33e7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f33e806f3a0 0x7f33e80b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48174/0 (socket says 172.21.15.131:48174) 2024-04-28T12:39:07.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.181+0000 7f33e7e00640 1 -- 172.21.15.131:0/2683581333 learned_addr learned my addr 172.21.15.131:0/2683581333 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:07.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.181+0000 7f33e7e00640 1 -- 172.21.15.131:0/2683581333 >> v2:172.21.15.157:3300/0 conn(0x7f33e815bd60 msgr2=0x7f33e80b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:07.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.181+0000 7f33ed400640 1 --2- 172.21.15.131:0/2683581333 >> v2:172.21.15.157:3300/0 conn(0x7f33e815bd60 0x7f33e80b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:07.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.181+0000 7f33e7e00640 1 --2- 172.21.15.131:0/2683581333 >> v2:172.21.15.157:3300/0 conn(0x7f33e815bd60 0x7f33e80b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.181+0000 7f33e7e00640 1 -- 172.21.15.131:0/2683581333 >> v1:172.21.15.157:6789/0 conn(0x7f33e815b980 legacy=0x7f33e806ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:07.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.181+0000 7f33e7e00640 1 -- 172.21.15.131:0/2683581333 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e815c580 con 0x7f33e806f3a0 2024-04-28T12:39:07.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.182+0000 7f33e7e00640 1 --2- 172.21.15.131:0/2683581333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33e806f3a0 0x7f33e80b4610 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f33e8072bd0 tx=0x7f33d802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278d9589e27383c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:07.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.182+0000 7f33e7400640 1 -- 172.21.15.131:0/2683581333 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8037070 con 0x7f33e806f3a0 2024-04-28T12:39:07.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.182+0000 7f33e7400640 1 -- 172.21.15.131:0/2683581333 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d8032440 con 0x7f33e806f3a0 2024-04-28T12:39:07.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.182+0000 7f33e7400640 1 -- 172.21.15.131:0/2683581333 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8032810 con 0x7f33e806f3a0 2024-04-28T12:39:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.183+0000 7f33ede00640 1 -- 172.21.15.131:0/2683581333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33e806f3a0 msgr2=0x7f33e80b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.183+0000 7f33ede00640 1 --2- 172.21.15.131:0/2683581333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33e806f3a0 0x7f33e80b4610 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f33e8072bd0 tx=0x7f33d802a230 comp rx=0 tx=0).stop 2024-04-28T12:39:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.183+0000 7f33ede00640 1 -- 172.21.15.131:0/2683581333 shutdown_connections 2024-04-28T12:39:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.183+0000 7f33ede00640 1 --2- 172.21.15.131:0/2683581333 >> v2:172.21.15.157:3300/0 conn(0x7f33e815bd60 0x7f33e80b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.183+0000 7f33ede00640 1 --2- 172.21.15.131:0/2683581333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33e806f3a0 0x7f33e80b4610 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.183+0000 7f33ede00640 1 -- 172.21.15.131:0/2683581333 >> 172.21.15.131:0/2683581333 conn(0x7f33e8068090 msgr2=0x7f33e80684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.183+0000 7f33ede00640 1 -- 172.21.15.131:0/2683581333 shutdown_connections 2024-04-28T12:39:07.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.183+0000 7f33ede00640 1 -- 172.21.15.131:0/2683581333 wait complete. 2024-04-28T12:39:07.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.185+0000 7f33ede00640 1 Processor -- start 2024-04-28T12:39:07.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.185+0000 7f33ede00640 1 -- start start 2024-04-28T12:39:07.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.186+0000 7f33ede00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33e806f3a0 0x7f33e80b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.186+0000 7f33ede00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e815b980 0x7f33e81f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.186+0000 7f33ede00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e815bd60 0x7f33e81555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.186+0000 7f33ede00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f33e80b52b0 con 0x7f33e806f3a0 2024-04-28T12:39:07.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.186+0000 7f33ede00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33e815cb40 con 0x7f33e815bd60 2024-04-28T12:39:07.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.186+0000 7f33ede00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f33e815f6b0 con 0x7f33e815b980 2024-04-28T12:39:07.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.187+0000 7f33eca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33e806f3a0 0x7f33e80b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:07.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.187+0000 7f33eca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33e806f3a0 0x7f33e80b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48184/0 (socket says 172.21.15.131:48184) 2024-04-28T12:39:07.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.187+0000 7f33eca00640 1 -- 172.21.15.131:0/1483434219 learned_addr learned my addr 172.21.15.131:0/1483434219 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:07.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.187+0000 7f33eca00640 1 -- 172.21.15.131:0/1483434219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e815b980 msgr2=0x7f33e81f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:07.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.187+0000 7f33eca00640 1 --2- 172.21.15.131:0/1483434219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e815b980 0x7f33e81f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.187+0000 7f33eca00640 1 -- 172.21.15.131:0/1483434219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e815bd60 msgr2=0x7f33e81555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:07.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.187+0000 7f33eca00640 1 --2- 172.21.15.131:0/1483434219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e815bd60 0x7f33e81555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.187+0000 7f33eca00640 1 -- 172.21.15.131:0/1483434219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e8158f70 con 0x7f33e806f3a0 2024-04-28T12:39:07.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.187+0000 7f33eca00640 1 --2- 172.21.15.131:0/1483434219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33e806f3a0 0x7f33e80b34d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00c440 tx=0x7f33dc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:07.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.188+0000 7f33e4c00640 1 -- 172.21.15.131:0/1483434219 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc017070 con 0x7f33e806f3a0 2024-04-28T12:39:07.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.188+0000 7f33e4c00640 1 -- 172.21.15.131:0/1483434219 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33dc0035f0 con 0x7f33e806f3a0 2024-04-28T12:39:07.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.188+0000 7f33e4c00640 1 -- 172.21.15.131:0/1483434219 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0039e0 con 0x7f33e806f3a0 2024-04-28T12:39:07.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.188+0000 7f33ede00640 1 -- 172.21.15.131:0/1483434219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e81f5370 con 0x7f33e806f3a0 2024-04-28T12:39:07.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.188+0000 7f33ede00640 1 -- 172.21.15.131:0/1483434219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e81f5930 con 0x7f33e806f3a0 2024-04-28T12:39:07.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.189+0000 7f33ede00640 1 -- 172.21.15.131:0/1483434219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e80506c0 con 0x7f33e806f3a0 2024-04-28T12:39:07.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.190+0000 7f33e4c00640 1 -- 172.21.15.131:0/1483434219 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33dc003b80 con 0x7f33e806f3a0 2024-04-28T12:39:07.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.191+0000 7f33e4c00640 1 --2- 172.21.15.131:0/1483434219 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f33cc06c8f0 0x7f33cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.191+0000 7f33e4c00640 1 -- 172.21.15.131:0/1483434219 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f33dc08ae00 con 0x7f33e806f3a0 2024-04-28T12:39:07.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.192+0000 7f33e7e00640 1 --2- 172.21.15.131:0/1483434219 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f33cc06c8f0 0x7f33cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:07.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.193+0000 7f33e7e00640 1 --2- 172.21.15.131:0/1483434219 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f33cc06c8f0 0x7f33cc06edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f33d8034000 tx=0x7f33d802c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:07.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.197+0000 7f33e4c00640 1 -- 172.21.15.131:0/1483434219 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f33dc059350 con 0x7f33e806f3a0 2024-04-28T12:39:07.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.396+0000 7f33ede00640 1 -- 172.21.15.131:0/1483434219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f33e815e010 con 0x7f33e806f3a0 2024-04-28T12:39:07.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.397+0000 7f33e4c00640 1 -- 172.21.15.131:0/1483434219 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f33dc0010c0 con 0x7f33e806f3a0 2024-04-28T12:39:07.394 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.400+0000 7f33ede00640 1 -- 172.21.15.131:0/1483434219 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f33cc06c8f0 msgr2=0x7f33cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.400+0000 7f33ede00640 1 --2- 172.21.15.131:0/1483434219 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f33cc06c8f0 0x7f33cc06edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f33d8034000 tx=0x7f33d802c000 comp rx=0 tx=0).stop 2024-04-28T12:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.400+0000 7f33ede00640 1 -- 172.21.15.131:0/1483434219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33e806f3a0 msgr2=0x7f33e80b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.400+0000 7f33ede00640 1 --2- 172.21.15.131:0/1483434219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33e806f3a0 0x7f33e80b34d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00c440 tx=0x7f33dc00c900 comp rx=0 tx=0).stop 2024-04-28T12:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.400+0000 7f33ede00640 1 -- 172.21.15.131:0/1483434219 shutdown_connections 2024-04-28T12:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.400+0000 7f33ede00640 1 --2- 172.21.15.131:0/1483434219 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f33cc06c8f0 0x7f33cc06edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.400+0000 7f33ede00640 1 --2- 172.21.15.131:0/1483434219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33e815bd60 0x7f33e81555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.400+0000 7f33ede00640 1 --2- 172.21.15.131:0/1483434219 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33e815b980 0x7f33e81f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.400+0000 7f33ede00640 1 --2- 172.21.15.131:0/1483434219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33e806f3a0 0x7f33e80b34d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.400+0000 7f33ede00640 1 -- 172.21.15.131:0/1483434219 >> 172.21.15.131:0/1483434219 conn(0x7f33e8068090 msgr2=0x7f33e8157530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.400+0000 7f33ede00640 1 -- 172.21.15.131:0/1483434219 shutdown_connections 2024-04-28T12:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.401+0000 7f33ede00640 1 -- 172.21.15.131:0/1483434219 wait complete. 2024-04-28T12:39:07.416 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-28T12:39:07.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.622+0000 7fc6ef000640 1 Processor -- start 2024-04-28T12:39:07.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.622+0000 7fc6ef000640 1 -- start start 2024-04-28T12:39:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.623+0000 7fc6ef000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6e8160ae0 0x7fc6e815df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.623+0000 7fc6ef000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6e8161170 con 0x7fc6e813fc90 2024-04-28T12:39:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.623+0000 7fc6ef000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6e8161450 con 0x7fc6e80d0da0 2024-04-28T12:39:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.623+0000 7fc6ef000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e8161a10 con 0x7fc6e8160ae0 2024-04-28T12:39:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.624+0000 7fc6e7e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc6e813fc90 0x7fc6e8142c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38148/0 (socket says 172.21.15.131:38148) 2024-04-28T12:39:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.624+0000 7fc6e7e00640 1 -- 172.21.15.131:0/1580776195 learned_addr learned my addr 172.21.15.131:0/1580776195 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:07.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.624+0000 7fc6e7400640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6e8160ae0 0x7fc6e815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:07.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.624+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2543673020 0 0) 0x7fc6e8161170 con 0x7fc6e813fc90 2024-04-28T12:39:07.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.624+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6cc003520 con 0x7fc6e813fc90 2024-04-28T12:39:07.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.624+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2380268089 0 0) 0x7fc6cc003520 con 0x7fc6e813fc90 2024-04-28T12:39:07.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.624+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6e8160ae0 msgr2=0x7fc6e815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:07.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.624+0000 7fc6e6a00640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6e8160ae0 0x7fc6e815df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.624+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 >> v1:172.21.15.157:6789/0 conn(0x7fc6e80d0da0 legacy=0x7fc6e80cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.624+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e8161730 con 0x7fc6e813fc90 2024-04-28T12:39:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fc6d8001670 con 0x7fc6e813fc90 2024-04-28T12:39:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 >> v1:172.21.15.131:6789/0 conn(0x7fc6e813fc90 legacy=0x7fc6e8142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e6a00640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6cc0039e0 0x7fc6cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e6a00640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc004800 0x7fc6cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e6a00640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6e8160ae0 0x7fc6cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc00baa0 con 0x7fc6cc0039e0 2024-04-28T12:39:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc00bc50 con 0x7fc6cc004800 2024-04-28T12:39:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc6cc00be50 con 0x7fc6e8160ae0 2024-04-28T12:39:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e7e00640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6cc0039e0 0x7fc6cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e7400640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc004800 0x7fc6cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6ed000640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6e8160ae0 0x7fc6cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e7e00640 1 -- 172.21.15.131:0/1580776195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6e8160ae0 msgr2=0x7fc6cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e7e00640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6e8160ae0 0x7fc6cc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e7e00640 1 -- 172.21.15.131:0/1580776195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc004800 msgr2=0x7fc6cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e7e00640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc004800 0x7fc6cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.625+0000 7fc6e7e00640 1 -- 172.21.15.131:0/1580776195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6cc003520 con 0x7fc6cc0039e0 2024-04-28T12:39:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.626+0000 7fc6e7400640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc004800 0x7fc6cc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.627+0000 7fc6e7e00640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6cc0039e0 0x7fc6cc003e00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c160 tx=0x7fc6cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d649b8da2aeaaaf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.627+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8007c60 con 0x7fc6cc0039e0 2024-04-28T12:39:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.627+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6d8005af0 con 0x7fc6cc0039e0 2024-04-28T12:39:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.627+0000 7fc6e6a00640 1 -- 172.21.15.131:0/1580776195 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8007070 con 0x7fc6cc0039e0 2024-04-28T12:39:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.627+0000 7fc6ef000640 1 -- 172.21.15.131:0/1580776195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6cc0039e0 msgr2=0x7fc6cc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.627+0000 7fc6ef000640 1 --2- 172.21.15.131:0/1580776195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6cc0039e0 0x7fc6cc003e00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c160 tx=0x7fc6cc005830 comp rx=0 tx=0).stop 2024-04-28T12:39:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.627+0000 7fc6ed000640 1 -- 172.21.15.131:0/1580776195 reap_dead start 2024-04-28T12:39:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.627+0000 7fc6ef000640 1 -- 172.21.15.131:0/1580776195 shutdown_connections 2024-04-28T12:39:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.627+0000 7fc6ef000640 1 -- 172.21.15.131:0/1580776195 >> 172.21.15.131:0/1580776195 conn(0x7fc6e80682a0 msgr2=0x7fc6e80686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.628+0000 7fc6ef000640 1 -- 172.21.15.131:0/1580776195 shutdown_connections 2024-04-28T12:39:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.628+0000 7fc6ef000640 1 -- 172.21.15.131:0/1580776195 wait complete. 2024-04-28T12:39:07.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.629+0000 7fc6ef000640 1 Processor -- start 2024-04-28T12:39:07.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.629+0000 7fc6ef000640 1 -- start start 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.630+0000 7fc6ef000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc0039e0 0x7fc6e8157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.630+0000 7fc6ef000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8155b90 0x7fc6e8159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.630+0000 7fc6ef000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6e8156010 0x7fc6e81595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.630+0000 7fc6ef000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e80d08c0 con 0x7fc6e8155b90 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.630+0000 7fc6ef000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8160f20 con 0x7fc6cc0039e0 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.630+0000 7fc6ef000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e81433a0 con 0x7fc6e8156010 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.631+0000 7fc6e7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8155b90 0x7fc6e8159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.631+0000 7fc6e7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8155b90 0x7fc6e8159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48208/0 (socket says 172.21.15.131:48208) 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.631+0000 7fc6e7400640 1 -- 172.21.15.131:0/3500404852 learned_addr learned my addr 172.21.15.131:0/3500404852 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.631+0000 7fc6e7e00640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc0039e0 0x7fc6e8157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.631+0000 7fc6e7400640 1 -- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6e8156010 msgr2=0x7fc6e81595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.631+0000 7fc6e7400640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6e8156010 0x7fc6e81595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.631+0000 7fc6e7400640 1 -- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc0039e0 msgr2=0x7fc6e8157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:07.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.631+0000 7fc6e7400640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc0039e0 0x7fc6e8157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.631+0000 7fc6e7400640 1 -- 172.21.15.131:0/3500404852 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e8156570 con 0x7fc6e8155b90 2024-04-28T12:39:07.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.631+0000 7fc6e7e00640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc0039e0 0x7fc6e8157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:07.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.631+0000 7fc6e7400640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8155b90 0x7fc6e8159060 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00108c0 tx=0x7fc6d0010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:07.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.632+0000 7fc6dfe00640 1 -- 172.21.15.131:0/3500404852 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d000f040 con 0x7fc6e8155b90 2024-04-28T12:39:07.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.632+0000 7fc6dfe00640 1 -- 172.21.15.131:0/3500404852 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6d000eb00 con 0x7fc6e8155b90 2024-04-28T12:39:07.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.632+0000 7fc6dfe00640 1 -- 172.21.15.131:0/3500404852 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d001c400 con 0x7fc6e8155b90 2024-04-28T12:39:07.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.632+0000 7fc6ef000640 1 -- 172.21.15.131:0/3500404852 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e8156850 con 0x7fc6e8155b90 2024-04-28T12:39:07.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.632+0000 7fc6ef000640 1 -- 172.21.15.131:0/3500404852 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e8156e10 con 0x7fc6e8155b90 2024-04-28T12:39:07.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.633+0000 7fc6ef000640 1 -- 172.21.15.131:0/3500404852 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e8069350 con 0x7fc6e8155b90 2024-04-28T12:39:07.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.634+0000 7fc6dfe00640 1 -- 172.21.15.131:0/3500404852 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6d001c5a0 con 0x7fc6e8155b90 2024-04-28T12:39:07.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.635+0000 7fc6dfe00640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc6c406c920 0x7fc6c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:07.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.635+0000 7fc6dfe00640 1 -- 172.21.15.131:0/3500404852 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fc6d008be90 con 0x7fc6e8155b90 2024-04-28T12:39:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.635+0000 7fc6e7e00640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc6c406c920 0x7fc6c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:07.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.636+0000 7fc6e7e00640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc6c406c920 0x7fc6c406ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0057e0 tx=0x7fc6d802d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:07.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.639+0000 7fc6dfe00640 1 -- 172.21.15.131:0/3500404852 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc6d0018030 con 0x7fc6e8155b90 2024-04-28T12:39:07.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.842+0000 7fc6ef000640 1 -- 172.21.15.131:0/3500404852 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc6e8073cb0 con 0x7fc6e8155b90 2024-04-28T12:39:07.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.843+0000 7fc6dfe00640 1 -- 172.21.15.131:0/3500404852 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7fc6d0059e30 con 0x7fc6e8155b90 2024-04-28T12:39:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:07.840 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-04-28T12:39:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-04-28T12:39:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-04-28T12:39:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-04-28T12:39:07.840 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:07.840 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-04-28T12:39:07.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.846+0000 7fc6ef000640 1 -- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc6c406c920 msgr2=0x7fc6c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:07.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.846+0000 7fc6ef000640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc6c406c920 0x7fc6c406ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0057e0 tx=0x7fc6d802d000 comp rx=0 tx=0).stop 2024-04-28T12:39:07.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.846+0000 7fc6ef000640 1 -- 172.21.15.131:0/3500404852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8155b90 msgr2=0x7fc6e8159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:07.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.846+0000 7fc6ef000640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8155b90 0x7fc6e8159060 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00108c0 tx=0x7fc6d0010d80 comp rx=0 tx=0).stop 2024-04-28T12:39:07.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.846+0000 7fc6ef000640 1 -- 172.21.15.131:0/3500404852 shutdown_connections 2024-04-28T12:39:07.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.846+0000 7fc6ef000640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc6c406c920 0x7fc6c406ede0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.846+0000 7fc6ef000640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6e8156010 0x7fc6e81595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.846+0000 7fc6ef000640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6e8155b90 0x7fc6e8159060 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.846+0000 7fc6ef000640 1 --2- 172.21.15.131:0/3500404852 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6cc0039e0 0x7fc6e8157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:07.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.846+0000 7fc6ef000640 1 -- 172.21.15.131:0/3500404852 >> 172.21.15.131:0/3500404852 conn(0x7fc6e80682a0 msgr2=0x7fc6e8140e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:07.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.846+0000 7fc6ef000640 1 -- 172.21.15.131:0/3500404852 shutdown_connections 2024-04-28T12:39:07.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:07.846+0000 7fc6ef000640 1 -- 172.21.15.131:0/3500404852 wait complete. 2024-04-28T12:39:07.862 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-28T12:39:07.863 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-28T12:39:08.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.074+0000 7f72dd200640 1 Processor -- start 2024-04-28T12:39:08.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.074+0000 7f72dd200640 1 -- start start 2024-04-28T12:39:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.076+0000 7f72dd200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f72d8162960 0x7f72d80d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.076+0000 7f72dd200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f72d80d1dd0 0x7f72d80c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.076+0000 7f72dd200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72d80d0dd0 con 0x7f72d8162580 2024-04-28T12:39:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.076+0000 7f72dd200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f72d80d5d30 con 0x7f72d80d1dd0 2024-04-28T12:39:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.076+0000 7f72dd200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f72d80d0530 con 0x7f72d8162960 2024-04-28T12:39:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.076+0000 7f72d5e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f72d8162580 0x7f72d80d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38154/0 (socket says 172.21.15.131:38154) 2024-04-28T12:39:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.076+0000 7f72d5e00640 1 -- 172.21.15.131:0/3715163405 learned_addr learned my addr 172.21.15.131:0/3715163405 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.076+0000 7f72d6800640 1 --2- 172.21.15.131:0/3715163405 >> v2:172.21.15.131:3300/0 conn(0x7f72d80d1dd0 0x7f72d80c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.077+0000 7f72cfe00640 1 --2- 172.21.15.131:0/3715163405 >> v2:172.21.15.157:3300/0 conn(0x7f72d8162960 0x7f72d80d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.077+0000 7f72d5400640 1 -- 172.21.15.131:0/3715163405 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3465560366 0 0) 0x7f72d80d0dd0 con 0x7f72d8162580 2024-04-28T12:39:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.077+0000 7f72d5400640 1 -- 172.21.15.131:0/3715163405 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72bc003520 con 0x7f72d8162580 2024-04-28T12:39:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.077+0000 7f72d6800640 1 -- 172.21.15.131:0/3715163405 >> v2:172.21.15.157:3300/0 conn(0x7f72d8162960 msgr2=0x7f72d80d5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.077+0000 7f72d6800640 1 --2- 172.21.15.131:0/3715163405 >> v2:172.21.15.157:3300/0 conn(0x7f72d8162960 0x7f72d80d5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.077+0000 7f72d6800640 1 -- 172.21.15.131:0/3715163405 >> v1:172.21.15.131:6789/0 conn(0x7f72d8162580 legacy=0x7f72d80d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.077+0000 7f72d6800640 1 -- 172.21.15.131:0/3715163405 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d80d0810 con 0x7f72d80d1dd0 2024-04-28T12:39:08.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.078+0000 7f72d6800640 1 --2- 172.21.15.131:0/3715163405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80d1dd0 0x7f72d80c8d40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f72d808f070 tx=0x7f72d002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47803385c56d8507 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:08.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.078+0000 7f72d5400640 1 -- 172.21.15.131:0/3715163405 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0007740 con 0x7f72d80d1dd0 2024-04-28T12:39:08.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.078+0000 7f72d5400640 1 -- 172.21.15.131:0/3715163405 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d00078e0 con 0x7f72d80d1dd0 2024-04-28T12:39:08.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.078+0000 7f72d5400640 1 -- 172.21.15.131:0/3715163405 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0007cb0 con 0x7f72d80d1dd0 2024-04-28T12:39:08.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.079+0000 7f72dd200640 1 -- 172.21.15.131:0/3715163405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80d1dd0 msgr2=0x7f72d80c8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:08.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.079+0000 7f72dd200640 1 --2- 172.21.15.131:0/3715163405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80d1dd0 0x7f72d80c8d40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f72d808f070 tx=0x7f72d002a680 comp rx=0 tx=0).stop 2024-04-28T12:39:08.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.079+0000 7f72dd200640 1 -- 172.21.15.131:0/3715163405 shutdown_connections 2024-04-28T12:39:08.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.079+0000 7f72dd200640 1 --2- 172.21.15.131:0/3715163405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d80d1dd0 0x7f72d80c8d40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:08.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.079+0000 7f72dd200640 1 --2- 172.21.15.131:0/3715163405 >> v2:172.21.15.157:3300/0 conn(0x7f72d8162960 0x7f72d80d5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:08.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.079+0000 7f72dd200640 1 -- 172.21.15.131:0/3715163405 >> 172.21.15.131:0/3715163405 conn(0x7f72d809e1e0 msgr2=0x7f72d80d4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:08.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.079+0000 7f72dd200640 1 -- 172.21.15.131:0/3715163405 shutdown_connections 2024-04-28T12:39:08.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.079+0000 7f72dd200640 1 -- 172.21.15.131:0/3715163405 wait complete. 2024-04-28T12:39:08.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.081+0000 7f72dd200640 1 Processor -- start 2024-04-28T12:39:08.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.081+0000 7f72dd200640 1 -- start start 2024-04-28T12:39:08.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.082+0000 7f72dd200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72d80d1dd0 0x7f72d815b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.082+0000 7f72dd200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d8162580 0x7f72d80cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.082+0000 7f72dd200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72d8162960 0x7f72d8157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.082+0000 7f72dd200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f72d80d2200 con 0x7f72d8162580 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.082+0000 7f72dd200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f72d80d2d00 con 0x7f72d8162960 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.082+0000 7f72dd200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f72d80d2380 con 0x7f72d80d1dd0 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.082+0000 7f72cfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d8162580 0x7f72d80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.082+0000 7f72cfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d8162580 0x7f72d80cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48226/0 (socket says 172.21.15.131:48226) 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.082+0000 7f72cfe00640 1 -- 172.21.15.131:0/2809213321 learned_addr learned my addr 172.21.15.131:0/2809213321 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.082+0000 7f72d5e00640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72d80d1dd0 0x7f72d815b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.082+0000 7f72d6800640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72d8162960 0x7f72d8157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.083+0000 7f72cfe00640 1 -- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72d80d1dd0 msgr2=0x7f72d815b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.083+0000 7f72cfe00640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72d80d1dd0 0x7f72d815b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.083+0000 7f72cfe00640 1 -- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72d8162960 msgr2=0x7f72d8157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:08.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.083+0000 7f72cfe00640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72d8162960 0x7f72d8157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:08.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.083+0000 7f72cfe00640 1 -- 172.21.15.131:0/2809213321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d80b8b20 con 0x7f72d8162580 2024-04-28T12:39:08.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.083+0000 7f72d5e00640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72d80d1dd0 0x7f72d815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:08.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.083+0000 7f72cfe00640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d8162580 0x7f72d80cc440 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f72c000d5f0 tx=0x7f72c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:08.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.083+0000 7f72ce000640 1 -- 172.21.15.131:0/2809213321 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c00125f0 con 0x7f72d8162580 2024-04-28T12:39:08.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.083+0000 7f72ce000640 1 -- 172.21.15.131:0/2809213321 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c0019070 con 0x7f72d8162580 2024-04-28T12:39:08.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.083+0000 7f72ce000640 1 -- 172.21.15.131:0/2809213321 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c00129d0 con 0x7f72d8162580 2024-04-28T12:39:08.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.083+0000 7f72dd200640 1 -- 172.21.15.131:0/2809213321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d80b8e00 con 0x7f72d8162580 2024-04-28T12:39:08.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.083+0000 7f72dd200640 1 -- 172.21.15.131:0/2809213321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d80b93c0 con 0x7f72d8162580 2024-04-28T12:39:08.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.085+0000 7f72dd200640 1 -- 172.21.15.131:0/2809213321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d8069a60 con 0x7f72d8162580 2024-04-28T12:39:08.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.085+0000 7f72ce000640 1 -- 172.21.15.131:0/2809213321 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72c0012b70 con 0x7f72d8162580 2024-04-28T12:39:08.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.086+0000 7f72ce000640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f72b406c8f0 0x7f72b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:08.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.087+0000 7f72ce000640 1 -- 172.21.15.131:0/2809213321 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f72c008bde0 con 0x7f72d8162580 2024-04-28T12:39:08.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.087+0000 7f72d5e00640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f72b406c8f0 0x7f72b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:08.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.088+0000 7f72d5e00640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f72b406c8f0 0x7f72b406edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f72c8005880 tx=0x7f72c8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:08.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.091+0000 7f72ce000640 1 -- 172.21.15.131:0/2809213321 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72c005a3d0 con 0x7f72d8162580 2024-04-28T12:39:08.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:08.296+0000 7f72dd200640 1 -- 172.21.15.131:0/2809213321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f72d80b7570 con 0x7f72d8162580 2024-04-28T12:39:09.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.042+0000 7f72ce000640 1 -- 172.21.15.131:0/2809213321 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7f72c0059cd0 con 0x7f72d8162580 2024-04-28T12:39:09.039 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-04-28T12:39:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.046+0000 7f72dd200640 1 -- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f72b406c8f0 msgr2=0x7f72b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.046+0000 7f72dd200640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f72b406c8f0 0x7f72b406edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f72c8005880 tx=0x7f72c8009040 comp rx=0 tx=0).stop 2024-04-28T12:39:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.046+0000 7f72dd200640 1 -- 172.21.15.131:0/2809213321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d8162580 msgr2=0x7f72d80cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.046+0000 7f72dd200640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d8162580 0x7f72d80cc440 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f72c000d5f0 tx=0x7f72c000dab0 comp rx=0 tx=0).stop 2024-04-28T12:39:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.046+0000 7f72dd200640 1 -- 172.21.15.131:0/2809213321 shutdown_connections 2024-04-28T12:39:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.046+0000 7f72dd200640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f72b406c8f0 0x7f72b406edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.046+0000 7f72dd200640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72d8162960 0x7f72d8157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.046+0000 7f72dd200640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d8162580 0x7f72d80cc440 secure :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f72c000d5f0 tx=0x7f72c000dab0 comp rx=0 tx=0).stop 2024-04-28T12:39:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.046+0000 7f72dd200640 1 --2- 172.21.15.131:0/2809213321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72d80d1dd0 0x7f72d815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:09.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.046+0000 7f72dd200640 1 -- 172.21.15.131:0/2809213321 >> 172.21.15.131:0/2809213321 conn(0x7f72d809e1e0 msgr2=0x7f72d80b66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.047+0000 7f72dd200640 1 -- 172.21.15.131:0/2809213321 shutdown_connections 2024-04-28T12:39:09.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.047+0000 7f72dd200640 1 -- 172.21.15.131:0/2809213321 wait complete. 2024-04-28T12:39:09.070 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-28T12:39:09.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.272+0000 7f0414200640 1 Processor -- start 2024-04-28T12:39:09.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.272+0000 7f0414200640 1 -- start start 2024-04-28T12:39:09.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.273+0000 7f0414200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f040c0b5100 0x7f040c15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:09.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.273+0000 7f0414200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f040c1602e0 con 0x7f040c15f7b0 2024-04-28T12:39:09.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.273+0000 7f0414200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f040c1605c0 con 0x7f040c159760 2024-04-28T12:39:09.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.273+0000 7f0414200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f040c0b4560 con 0x7f040c0b5100 2024-04-28T12:39:09.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.274+0000 7f0410e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f040c15f7b0 0x7f040c0b49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38168/0 (socket says 172.21.15.131:38168) 2024-04-28T12:39:09.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.274+0000 7f0410e00640 1 -- 172.21.15.131:0/4015080437 learned_addr learned my addr 172.21.15.131:0/4015080437 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:09.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.274+0000 7f0412200640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f040c0b5100 0x7f040c15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:09.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.274+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1074903434 0 0) 0x7f040c1602e0 con 0x7f040c15f7b0 2024-04-28T12:39:09.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.274+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03f4003520 con 0x7f040c15f7b0 2024-04-28T12:39:09.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 872434781 0 0) 0x7f03f4003520 con 0x7f040c15f7b0 2024-04-28T12:39:09.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f040c0b5100 msgr2=0x7f040c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:09.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f040c0b5100 0x7f040c15fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 >> v1:172.21.15.157:6789/0 conn(0x7f040c159760 legacy=0x7f040c0ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c0b41f0 con 0x7f040c15f7b0 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f03f8001690 con 0x7f040c15f7b0 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 >> v1:172.21.15.131:6789/0 conn(0x7f040c15f7b0 legacy=0x7f040c0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f040c0b5100 0x7f03f40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03f4003f60 0x7f03f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03f4004d40 0x7f03f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f040c0b4560 con 0x7f03f4004d40 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03f400baa0 con 0x7f03f4003f60 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f03f400bca0 con 0x7f040c0b5100 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f0412200640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03f4004d40 0x7f03f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f0410e00640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03f4003f60 0x7f03f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:09.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.275+0000 7f0411800640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f040c0b5100 0x7f03f40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:09.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.276+0000 7f0412200640 1 -- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f040c0b5100 msgr2=0x7f03f40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:09.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.276+0000 7f0412200640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f040c0b5100 0x7f03f40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:09.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.276+0000 7f0412200640 1 -- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03f4003f60 msgr2=0x7f03f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:09.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.276+0000 7f0412200640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03f4003f60 0x7f03f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:09.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.276+0000 7f0412200640 1 -- 172.21.15.131:0/4015080437 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f4003520 con 0x7f03f4004d40 2024-04-28T12:39:09.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.276+0000 7f0411800640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f040c0b5100 0x7f03f40039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:09.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.276+0000 7f0410e00640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03f4003f60 0x7f03f4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:09.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.277+0000 7f0412200640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03f4004d40 0x7f03f40051c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0404009f90 tx=0x7f03f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51049cc0478d2ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:09.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.277+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404009c10 con 0x7f03f4004d40 2024-04-28T12:39:09.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.277+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0404036070 con 0x7f03f4004d40 2024-04-28T12:39:09.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.277+0000 7f040be00640 1 -- 172.21.15.131:0/4015080437 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404031850 con 0x7f03f4004d40 2024-04-28T12:39:09.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.277+0000 7f0414200640 1 -- 172.21.15.131:0/4015080437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03f4004d40 msgr2=0x7f03f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:09.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.277+0000 7f0414200640 1 --2- 172.21.15.131:0/4015080437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03f4004d40 0x7f03f40051c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0404009f90 tx=0x7f03f4005830 comp rx=0 tx=0).stop 2024-04-28T12:39:09.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.277+0000 7f0412200640 1 -- 172.21.15.131:0/4015080437 reap_dead start 2024-04-28T12:39:09.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.278+0000 7f0414200640 1 -- 172.21.15.131:0/4015080437 shutdown_connections 2024-04-28T12:39:09.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.278+0000 7f0414200640 1 -- 172.21.15.131:0/4015080437 >> 172.21.15.131:0/4015080437 conn(0x7f040c0b2890 msgr2=0x7f040c0b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:09.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.278+0000 7f0414200640 1 -- 172.21.15.131:0/4015080437 shutdown_connections 2024-04-28T12:39:09.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.278+0000 7f0414200640 1 -- 172.21.15.131:0/4015080437 wait complete. 2024-04-28T12:39:09.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.279+0000 7f0414200640 1 Processor -- start 2024-04-28T12:39:09.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.279+0000 7f0414200640 1 -- start start 2024-04-28T12:39:09.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.280+0000 7f0414200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03f4004d40 0x7f040c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.280+0000 7f0414200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c15a190 0x7f040c0d0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.280+0000 7f0414200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f040c157750 0x7f040c148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.280+0000 7f0414200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f040c0720a0 con 0x7f040c157750 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.280+0000 7f0414200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f040c0b3ee0 con 0x7f040c15a190 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.280+0000 7f0414200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f040c159b40 con 0x7f03f4004d40 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.281+0000 7f0412200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f040c157750 0x7f040c148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.281+0000 7f0412200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f040c157750 0x7f040c148340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48240/0 (socket says 172.21.15.131:48240) 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.281+0000 7f0412200640 1 -- 172.21.15.131:0/2038225174 learned_addr learned my addr 172.21.15.131:0/2038225174 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.281+0000 7f0410e00640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c15a190 0x7f040c0d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.281+0000 7f0411800640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03f4004d40 0x7f040c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.281+0000 7f0412200640 1 -- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03f4004d40 msgr2=0x7f040c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.281+0000 7f0412200640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03f4004d40 0x7f040c0ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.281+0000 7f0412200640 1 -- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c15a190 msgr2=0x7f040c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:09.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.281+0000 7f0412200640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c15a190 0x7f040c0d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:09.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.281+0000 7f0412200640 1 -- 172.21.15.131:0/2038225174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c15a5f0 con 0x7f040c157750 2024-04-28T12:39:09.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.281+0000 7f0412200640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f040c157750 0x7f040c148340 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f040400e230 tx=0x7f0404009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:09.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.282+0000 7f0409600640 1 -- 172.21.15.131:0/2038225174 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404009c10 con 0x7f040c157750 2024-04-28T12:39:09.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.282+0000 7f0409600640 1 -- 172.21.15.131:0/2038225174 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0404003990 con 0x7f040c157750 2024-04-28T12:39:09.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.282+0000 7f0409600640 1 -- 172.21.15.131:0/2038225174 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404003d80 con 0x7f040c157750 2024-04-28T12:39:09.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.282+0000 7f0414200640 1 -- 172.21.15.131:0/2038225174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040c0cf3a0 con 0x7f040c157750 2024-04-28T12:39:09.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.282+0000 7f0414200640 1 -- 172.21.15.131:0/2038225174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040c0cf960 con 0x7f040c157750 2024-04-28T12:39:09.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.283+0000 7f0414200640 1 -- 172.21.15.131:0/2038225174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f040c069350 con 0x7f040c157750 2024-04-28T12:39:09.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.284+0000 7f0409600640 1 -- 172.21.15.131:0/2038225174 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0404005d10 con 0x7f040c157750 2024-04-28T12:39:09.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.285+0000 7f0409600640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f03ec06c8f0 0x7f03ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:09.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.285+0000 7f0409600640 1 -- 172.21.15.131:0/2038225174 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7f04040ac9e0 con 0x7f040c157750 2024-04-28T12:39:09.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.286+0000 7f0411800640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f03ec06c8f0 0x7f03ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:09.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.287+0000 7f0411800640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f03ec06c8f0 0x7f03ec06edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0400001630 tx=0x7f0400009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:09.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.290+0000 7f0409600640 1 -- 172.21.15.131:0/2038225174 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f040402e050 con 0x7f040c157750 2024-04-28T12:39:09.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:09.492+0000 7f0414200640 1 -- 172.21.15.131:0/2038225174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f040c158bc0 con 0x7f040c157750 2024-04-28T12:39:10.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.057+0000 7f0409600640 1 -- 172.21.15.131:0/2038225174 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7f040407a6e0 con 0x7f040c157750 2024-04-28T12:39:10.055 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-04-28T12:39:10.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.065+0000 7f0414200640 1 -- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f03ec06c8f0 msgr2=0x7f03ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:10.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.065+0000 7f0414200640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f03ec06c8f0 0x7f03ec06edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0400001630 tx=0x7f0400009040 comp rx=0 tx=0).stop 2024-04-28T12:39:10.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.065+0000 7f0414200640 1 -- 172.21.15.131:0/2038225174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f040c157750 msgr2=0x7f040c148340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:10.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.065+0000 7f0414200640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f040c157750 0x7f040c148340 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f040400e230 tx=0x7f0404009750 comp rx=0 tx=0).stop 2024-04-28T12:39:10.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.065+0000 7f0414200640 1 -- 172.21.15.131:0/2038225174 shutdown_connections 2024-04-28T12:39:10.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.065+0000 7f0414200640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f03ec06c8f0 0x7f03ec06edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:10.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.065+0000 7f0414200640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f040c157750 0x7f040c148340 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:10.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.065+0000 7f0414200640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f040c15a190 0x7f040c0d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:10.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.065+0000 7f0414200640 1 --2- 172.21.15.131:0/2038225174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f03f4004d40 0x7f040c0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:10.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.065+0000 7f0414200640 1 -- 172.21.15.131:0/2038225174 >> 172.21.15.131:0/2038225174 conn(0x7f040c0b2890 msgr2=0x7f040c0b5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:10.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.070+0000 7f0414200640 1 -- 172.21.15.131:0/2038225174 shutdown_connections 2024-04-28T12:39:10.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.073+0000 7f0414200640 1 -- 172.21.15.131:0/2038225174 wait complete. 2024-04-28T12:39:10.101 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-28T12:39:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.279+0000 7f6ed1e00640 1 Processor -- start 2024-04-28T12:39:10.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.279+0000 7f6ed1e00640 1 -- start start 2024-04-28T12:39:10.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.281+0000 7f6ed1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ecc15a7e0 0x7f6ecc0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:10.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.281+0000 7f6ed1e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ecc15d9c0 0x7f6ecc0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:10.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.281+0000 7f6ed1e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ecc0b3900 con 0x7f6ecc0b5410 2024-04-28T12:39:10.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.281+0000 7f6ed1e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f6ecc15abc0 con 0x7f6ecc15d9c0 2024-04-28T12:39:10.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.281+0000 7f6ed1e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6ecc0b3b70 con 0x7f6ecc15a7e0 2024-04-28T12:39:10.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.282+0000 7f6ed1400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ecc15d9c0 0x7f6ecc0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:10.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.282+0000 7f6ed0a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ecc15a7e0 0x7f6ecc0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:10.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.282+0000 7f6ed1400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f6ecc15d9c0 0x7f6ecc0b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:59286/0 (socket says 172.21.15.131:59286) 2024-04-28T12:39:10.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.282+0000 7f6ed1400640 1 -- 172.21.15.131:0/2473905033 learned_addr learned my addr 172.21.15.131:0/2473905033 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:10.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.282+0000 7f6ecbe00640 1 --1- 172.21.15.131:0/2473905033 >> v1:172.21.15.157:6789/0 conn(0x7f6ecc0b5410 0x7f6ecc15d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.131:50458/0 (socket says 172.21.15.131:50458) 2024-04-28T12:39:10.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.283+0000 7f6ed1400640 1 -- 172.21.15.131:0/2473905033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ecc15a7e0 msgr2=0x7f6ecc0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:10.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.283+0000 7f6ed1400640 1 --2- 172.21.15.131:0/2473905033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ecc15a7e0 0x7f6ecc0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:10.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.283+0000 7f6ed1400640 1 -- 172.21.15.131:0/2473905033 >> v1:172.21.15.157:6789/0 conn(0x7f6ecc0b5410 legacy=0x7f6ecc15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:10.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.283+0000 7f6ed1400640 1 -- 172.21.15.131:0/2473905033 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc0b3e50 con 0x7f6ecc15d9c0 2024-04-28T12:39:10.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.283+0000 7f6ed0a00640 1 --2- 172.21.15.131:0/2473905033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ecc15a7e0 0x7f6ecc0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:39:10.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.284+0000 7f6ed1400640 1 --2- 172.21.15.131:0/2473905033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ecc15d9c0 0x7f6ecc0b5890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009e50 tx=0x7f6ec402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8ba4223f5b2f934 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:10.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.284+0000 7f6ecb400640 1 -- 172.21.15.131:0/2473905033 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4036070 con 0x7f6ecc15d9c0 2024-04-28T12:39:10.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.284+0000 7f6ecb400640 1 -- 172.21.15.131:0/2473905033 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec4031600 con 0x7f6ecc15d9c0 2024-04-28T12:39:10.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.284+0000 7f6ecb400640 1 -- 172.21.15.131:0/2473905033 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec40319d0 con 0x7f6ecc15d9c0 2024-04-28T12:39:10.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.285+0000 7f6ed1e00640 1 -- 172.21.15.131:0/2473905033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ecc15d9c0 msgr2=0x7f6ecc0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:10.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.285+0000 7f6ed1e00640 1 --2- 172.21.15.131:0/2473905033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ecc15d9c0 0x7f6ecc0b5890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009e50 tx=0x7f6ec402a710 comp rx=0 tx=0).stop 2024-04-28T12:39:10.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.285+0000 7f6ed1e00640 1 -- 172.21.15.131:0/2473905033 shutdown_connections 2024-04-28T12:39:10.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.285+0000 7f6ed1e00640 1 --2- 172.21.15.131:0/2473905033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ecc15d9c0 0x7f6ecc0b5890 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:10.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.285+0000 7f6ed1e00640 1 --2- 172.21.15.131:0/2473905033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ecc15a7e0 0x7f6ecc0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:10.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.285+0000 7f6ed1e00640 1 -- 172.21.15.131:0/2473905033 >> 172.21.15.131:0/2473905033 conn(0x7f6ecc068090 msgr2=0x7f6ecc0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:10.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.285+0000 7f6ed1e00640 1 -- 172.21.15.131:0/2473905033 shutdown_connections 2024-04-28T12:39:10.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.285+0000 7f6ed1e00640 1 -- 172.21.15.131:0/2473905033 wait complete. 2024-04-28T12:39:10.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.287+0000 7f6ed1e00640 1 Processor -- start 2024-04-28T12:39:10.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.287+0000 7f6ed1e00640 1 -- start start 2024-04-28T12:39:10.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.288+0000 7f6ed1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc0b5410 0x7f6ecc0b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:10.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.288+0000 7f6ed1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ecc15a7e0 0x7f6ecc158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.288+0000 7f6ed1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ecc15d9c0 0x7f6ecc159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.288+0000 7f6ed1e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc0b4b90 con 0x7f6ecc0b5410 2024-04-28T12:39:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.288+0000 7f6ed1e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc15e010 con 0x7f6ecc15d9c0 2024-04-28T12:39:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.288+0000 7f6ed1e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6ecc15e2e0 con 0x7f6ecc15a7e0 2024-04-28T12:39:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.288+0000 7f6ed0a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc0b5410 0x7f6ecc0b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.288+0000 7f6ed0a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc0b5410 0x7f6ecc0b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48250/0 (socket says 172.21.15.131:48250) 2024-04-28T12:39:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.288+0000 7f6ed0a00640 1 -- 172.21.15.131:0/490100364 learned_addr learned my addr 172.21.15.131:0/490100364 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.289+0000 7f6ecbe00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ecc15a7e0 0x7f6ecc158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.289+0000 7f6ed1400640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ecc15d9c0 0x7f6ecc159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.289+0000 7f6ed0a00640 1 -- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ecc15a7e0 msgr2=0x7f6ecc158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.289+0000 7f6ed0a00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ecc15a7e0 0x7f6ecc158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.289+0000 7f6ed0a00640 1 -- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ecc15d9c0 msgr2=0x7f6ecc159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:10.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.289+0000 7f6ed0a00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ecc15d9c0 0x7f6ecc159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:10.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.289+0000 7f6ed0a00640 1 -- 172.21.15.131:0/490100364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc1f45a0 con 0x7f6ecc0b5410 2024-04-28T12:39:10.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.289+0000 7f6ed1400640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ecc15d9c0 0x7f6ecc159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:10.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.289+0000 7f6ed0a00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc0b5410 0x7f6ecc0b2a00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000ebe0 tx=0x7f6ec000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:10.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.290+0000 7f6ec8c00640 1 -- 172.21.15.131:0/490100364 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0017070 con 0x7f6ecc0b5410 2024-04-28T12:39:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.290+0000 7f6ec8c00640 1 -- 172.21.15.131:0/490100364 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec000cc30 con 0x7f6ecc0b5410 2024-04-28T12:39:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.290+0000 7f6ec8c00640 1 -- 172.21.15.131:0/490100364 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0013470 con 0x7f6ecc0b5410 2024-04-28T12:39:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.290+0000 7f6ed1e00640 1 -- 172.21.15.131:0/490100364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc0b3510 con 0x7f6ecc0b5410 2024-04-28T12:39:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.290+0000 7f6ed1e00640 1 -- 172.21.15.131:0/490100364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc1f4b20 con 0x7f6ecc0b5410 2024-04-28T12:39:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.291+0000 7f6ed1e00640 1 -- 172.21.15.131:0/490100364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ecc0506c0 con 0x7f6ecc0b5410 2024-04-28T12:39:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.292+0000 7f6ec8c00640 1 -- 172.21.15.131:0/490100364 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ec001a070 con 0x7f6ecc0b5410 2024-04-28T12:39:10.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.293+0000 7f6ec8c00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6eac06c8f0 0x7f6eac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.293+0000 7f6ec8c00640 1 -- 172.21.15.131:0/490100364 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7f6ec008bb50 con 0x7f6ecc0b5410 2024-04-28T12:39:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.294+0000 7f6ecbe00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6eac06c8f0 0x7f6eac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.295+0000 7f6ecbe00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6eac06c8f0 0x7f6eac06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc1f7b40 tx=0x7f6eb8007cc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.298+0000 7f6ec8c00640 1 -- 172.21.15.131:0/490100364 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ec0059ee0 con 0x7f6ecc0b5410 2024-04-28T12:39:10.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:10.532+0000 7f6ed1e00640 1 -- 172.21.15.131:0/490100364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6ecc1f4f00 con 0x7f6ecc0b5410 2024-04-28T12:39:11.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.062+0000 7f6ec8c00640 1 -- 172.21.15.131:0/490100364 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f6ec00597e0 con 0x7f6ecc0b5410 2024-04-28T12:39:11.059 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 5 and data pool 6 2024-04-28T12:39:11.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.067+0000 7f6ed1e00640 1 -- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6eac06c8f0 msgr2=0x7f6eac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.067+0000 7f6ed1e00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6eac06c8f0 0x7f6eac06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc1f7b40 tx=0x7f6eb8007cc0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.067+0000 7f6ed1e00640 1 -- 172.21.15.131:0/490100364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc0b5410 msgr2=0x7f6ecc0b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.067+0000 7f6ed1e00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc0b5410 0x7f6ecc0b2a00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000ebe0 tx=0x7f6ec000c520 comp rx=0 tx=0).stop 2024-04-28T12:39:11.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.067+0000 7f6ed1e00640 1 -- 172.21.15.131:0/490100364 shutdown_connections 2024-04-28T12:39:11.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.067+0000 7f6ed1e00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f6eac06c8f0 0x7f6eac06edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.067+0000 7f6ed1e00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ecc15d9c0 0x7f6ecc159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.067+0000 7f6ed1e00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ecc15a7e0 0x7f6ecc158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.067+0000 7f6ed1e00640 1 --2- 172.21.15.131:0/490100364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc0b5410 0x7f6ecc0b2a00 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.067+0000 7f6ed1e00640 1 -- 172.21.15.131:0/490100364 >> 172.21.15.131:0/490100364 conn(0x7f6ecc068090 msgr2=0x7f6ecc0729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:11.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.069+0000 7f6ed1e00640 1 -- 172.21.15.131:0/490100364 shutdown_connections 2024-04-28T12:39:11.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.070+0000 7f6ed1e00640 1 -- 172.21.15.131:0/490100364 wait complete. 2024-04-28T12:39:11.107 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:39:11.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.274+0000 7f46d0200640 1 Processor -- start 2024-04-28T12:39:11.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.274+0000 7f46d0200640 1 -- start start 2024-04-28T12:39:11.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.276+0000 7f46d0200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f46c8072b70 0x7f46c80d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.276+0000 7f46d0200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c8146ba0 con 0x7f46c80d1060 2024-04-28T12:39:11.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.276+0000 7f46d0200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c80731c0 con 0x7f46c81467c0 2024-04-28T12:39:11.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.276+0000 7f46d0200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f46c8073dd0 con 0x7f46c8072b70 2024-04-28T12:39:11.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.276+0000 7f46ce200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f46c80d1060 0x7f46c806dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38708/0 (socket says 172.21.15.131:38708) 2024-04-28T12:39:11.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.276+0000 7f46ce200640 1 -- 172.21.15.131:0/1735206890 learned_addr learned my addr 172.21.15.131:0/1735206890 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:11.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.276+0000 7f46cce00640 1 --2- 172.21.15.131:0/1735206890 >> v2:172.21.15.157:3300/0 conn(0x7f46c8072b70 0x7f46c80d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.276+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 102002406 0 0) 0x7f46c8146ba0 con 0x7f46c80d1060 2024-04-28T12:39:11.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.276+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46b0003520 con 0x7f46c80d1060 2024-04-28T12:39:11.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1903405470 0 0) 0x7f46b0003520 con 0x7f46c80d1060 2024-04-28T12:39:11.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 >> v2:172.21.15.157:3300/0 conn(0x7f46c8072b70 msgr2=0x7f46c80d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:11.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 --2- 172.21.15.131:0/1735206890 >> v2:172.21.15.157:3300/0 conn(0x7f46c8072b70 0x7f46c80d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 >> v1:172.21.15.157:6789/0 conn(0x7f46c81467c0 legacy=0x7f46c80b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c80734a0 con 0x7f46c80d1060 2024-04-28T12:39:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f46c00019f0 con 0x7f46c80d1060 2024-04-28T12:39:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 >> v1:172.21.15.131:6789/0 conn(0x7f46c80d1060 legacy=0x7f46c806dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 --2- 172.21.15.131:0/1735206890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46b00039b0 0x7f46b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 --2- 172.21.15.131:0/1735206890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46b0004800 0x7f46b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 --2- 172.21.15.131:0/1735206890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46b00051c0 0x7f46b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f46c8073dd0 con 0x7f46b00051c0 2024-04-28T12:39:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46b000c060 con 0x7f46b00039b0 2024-04-28T12:39:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f46b000c260 con 0x7f46b0004800 2024-04-28T12:39:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.277+0000 7f46ce200640 1 --2- 172.21.15.131:0/1735206890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46b00051c0 0x7f46b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.278+0000 7f46cd800640 1 --2- 172.21.15.131:0/1735206890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46b00039b0 0x7f46b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.278+0000 7f46cce00640 1 --2- 172.21.15.131:0/1735206890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46b0004800 0x7f46b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.278+0000 7f46ce200640 1 -- 172.21.15.131:0/1735206890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46b0004800 msgr2=0x7f46b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:11.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.278+0000 7f46ce200640 1 --2- 172.21.15.131:0/1735206890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46b0004800 0x7f46b0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.278+0000 7f46ce200640 1 -- 172.21.15.131:0/1735206890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46b00039b0 msgr2=0x7f46b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:11.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.278+0000 7f46ce200640 1 --2- 172.21.15.131:0/1735206890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46b00039b0 0x7f46b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.278+0000 7f46ce200640 1 -- 172.21.15.131:0/1735206890 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b0003520 con 0x7f46b00051c0 2024-04-28T12:39:11.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.279+0000 7f46ce200640 1 -- 172.21.15.131:0/1735206890 reap_dead start 2024-04-28T12:39:11.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.279+0000 7f46ce200640 1 --2- 172.21.15.131:0/1735206890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46b00051c0 0x7f46b000b9c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f46c000a860 tx=0x7f46b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4245d3aae6ca9e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:11.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.279+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c000a480 con 0x7f46b00051c0 2024-04-28T12:39:11.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.279+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c002f070 con 0x7f46b00051c0 2024-04-28T12:39:11.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.279+0000 7f46c7e00640 1 -- 172.21.15.131:0/1735206890 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c002bca0 con 0x7f46b00051c0 2024-04-28T12:39:11.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.280+0000 7f46d0200640 1 -- 172.21.15.131:0/1735206890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46b00051c0 msgr2=0x7f46b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.280+0000 7f46d0200640 1 --2- 172.21.15.131:0/1735206890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46b00051c0 0x7f46b000b9c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f46c000a860 tx=0x7f46b0005830 comp rx=0 tx=0).stop 2024-04-28T12:39:11.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.280+0000 7f46d0200640 1 -- 172.21.15.131:0/1735206890 shutdown_connections 2024-04-28T12:39:11.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.280+0000 7f46d0200640 1 --2- 172.21.15.131:0/1735206890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46b00051c0 0x7f46b000b9c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.280+0000 7f46d0200640 1 -- 172.21.15.131:0/1735206890 >> 172.21.15.131:0/1735206890 conn(0x7f46c80b2900 msgr2=0x7f46c80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:11.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.280+0000 7f46d0200640 1 -- 172.21.15.131:0/1735206890 shutdown_connections 2024-04-28T12:39:11.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.280+0000 7f46d0200640 1 -- 172.21.15.131:0/1735206890 wait complete. 2024-04-28T12:39:11.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.281+0000 7f46d0200640 1 Processor -- start 2024-04-28T12:39:11.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.281+0000 7f46d0200640 1 -- start start 2024-04-28T12:39:11.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.282+0000 7f46d0200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46b00051c0 0x7f46c813d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.282+0000 7f46d0200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c815c520 0x7f46c815f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.282+0000 7f46d0200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c815bf30 0x7f46c815fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.282+0000 7f46d0200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f46c80d0b20 con 0x7f46c815bf30 2024-04-28T12:39:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.282+0000 7f46d0200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46c8073a60 con 0x7f46c815c520 2024-04-28T12:39:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.282+0000 7f46d0200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f46c80d0ca0 con 0x7f46b00051c0 2024-04-28T12:39:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.283+0000 7f46ce200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c815bf30 0x7f46c815fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.283+0000 7f46ce200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c815bf30 0x7f46c815fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57222/0 (socket says 172.21.15.131:57222) 2024-04-28T12:39:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.283+0000 7f46ce200640 1 -- 172.21.15.131:0/4089355028 learned_addr learned my addr 172.21.15.131:0/4089355028 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.283+0000 7f46ce200640 1 -- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46b00051c0 msgr2=0x7f46c813d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.283+0000 7f46ce200640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46b00051c0 0x7f46c813d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.283+0000 7f46cce00640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c815c520 0x7f46c815f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.283+0000 7f46ce200640 1 -- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c815c520 msgr2=0x7f46c815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.283+0000 7f46ce200640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c815c520 0x7f46c815f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.283+0000 7f46ce200640 1 -- 172.21.15.131:0/4089355028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c815e0f0 con 0x7f46c815bf30 2024-04-28T12:39:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.283+0000 7f46ce200640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c815bf30 0x7f46c815fd90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f46c002b530 tx=0x7f46c002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.284+0000 7f46c5600640 1 -- 172.21.15.131:0/4089355028 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c000a480 con 0x7f46c815bf30 2024-04-28T12:39:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.284+0000 7f46c5600640 1 -- 172.21.15.131:0/4089355028 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c0034670 con 0x7f46c815bf30 2024-04-28T12:39:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.284+0000 7f46c5600640 1 -- 172.21.15.131:0/4089355028 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0034a60 con 0x7f46c815bf30 2024-04-28T12:39:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.284+0000 7f46d0200640 1 -- 172.21.15.131:0/4089355028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c813daf0 con 0x7f46c815bf30 2024-04-28T12:39:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.284+0000 7f46d0200640 1 -- 172.21.15.131:0/4089355028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c815dd30 con 0x7f46c815bf30 2024-04-28T12:39:11.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.285+0000 7f46d0200640 1 -- 172.21.15.131:0/4089355028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c8069350 con 0x7f46c815bf30 2024-04-28T12:39:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.286+0000 7f46c5600640 1 -- 172.21.15.131:0/4089355028 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46c0034c00 con 0x7f46c815bf30 2024-04-28T12:39:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.287+0000 7f46c5600640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f46a806c8f0 0x7f46a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.287+0000 7f46c5600640 1 -- 172.21.15.131:0/4089355028 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f46c00ac3e0 con 0x7f46c815bf30 2024-04-28T12:39:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.288+0000 7f46cd800640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f46a806c8f0 0x7f46a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.289+0000 7f46cd800640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f46a806c8f0 0x7f46a806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0014f0 tx=0x7f46bc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:11.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.292+0000 7f46c5600640 1 -- 172.21.15.131:0/4089355028 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46c0046060 con 0x7f46c815bf30 2024-04-28T12:39:11.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.492+0000 7f46d0200640 1 -- 172.21.15.131:0/4089355028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46c80ce730 con 0x7f46c815bf30 2024-04-28T12:39:11.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.494+0000 7f46c5600640 1 -- 172.21.15.131:0/4089355028 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16979 (secure 0 0 0) 0x7f46c007a5e0 con 0x7f46c815bf30 2024-04-28T12:39:11.491 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:11.491 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":37,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:39:10.533721+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:39:08.310817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-28T12:39:09.505586+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:39:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.497+0000 7f46d0200640 1 -- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f46a806c8f0 msgr2=0x7f46a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.497+0000 7f46d0200640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f46a806c8f0 0x7f46a806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0014f0 tx=0x7f46bc009040 comp rx=0 tx=0).stop 2024-04-28T12:39:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.497+0000 7f46d0200640 1 -- 172.21.15.131:0/4089355028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c815bf30 msgr2=0x7f46c815fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.497+0000 7f46d0200640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c815bf30 0x7f46c815fd90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f46c002b530 tx=0x7f46c002d070 comp rx=0 tx=0).stop 2024-04-28T12:39:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.497+0000 7f46d0200640 1 -- 172.21.15.131:0/4089355028 shutdown_connections 2024-04-28T12:39:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.497+0000 7f46d0200640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f46a806c8f0 0x7f46a806edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.497+0000 7f46d0200640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46c815bf30 0x7f46c815fd90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.497+0000 7f46d0200640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c815c520 0x7f46c815f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.497+0000 7f46d0200640 1 --2- 172.21.15.131:0/4089355028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46b00051c0 0x7f46c813d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.497+0000 7f46d0200640 1 -- 172.21.15.131:0/4089355028 >> 172.21.15.131:0/4089355028 conn(0x7f46c80b2900 msgr2=0x7f46c80b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.497+0000 7f46d0200640 1 -- 172.21.15.131:0/4089355028 shutdown_connections 2024-04-28T12:39:11.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.497+0000 7f46d0200640 1 -- 172.21.15.131:0/4089355028 wait complete. 2024-04-28T12:39:11.513 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:39:11.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.702+0000 7ff9b5e00640 1 Processor -- start 2024-04-28T12:39:11.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.702+0000 7ff9b5e00640 1 -- start start 2024-04-28T12:39:11.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.704+0000 7ff9b5e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff9b0070300 0x7ff9b0070a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.704+0000 7ff9b5e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff9b0070fb0 0x7ff9b015ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.704+0000 7ff9b5e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b00713c0 con 0x7ff9b00cefb0 2024-04-28T12:39:11.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.704+0000 7ff9b5e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b00d0750 con 0x7ff9b0070300 2024-04-28T12:39:11.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.704+0000 7ff9b5e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b00ce410 con 0x7ff9b0070fb0 2024-04-28T12:39:11.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.704+0000 7ff9aea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff9b00cefb0 0x7ff9b00d0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38718/0 (socket says 172.21.15.131:38718) 2024-04-28T12:39:11.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.704+0000 7ff9aea00640 1 -- 172.21.15.131:0/2868642207 learned_addr learned my addr 172.21.15.131:0/2868642207 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:11.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.704+0000 7ff9ae000640 1 --2- 172.21.15.131:0/2868642207 >> v2:172.21.15.131:3300/0 conn(0x7ff9b0070300 0x7ff9b0070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.705+0000 7ff9af400640 1 --2- 172.21.15.131:0/2868642207 >> v2:172.21.15.157:3300/0 conn(0x7ff9b0070fb0 0x7ff9b015ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.705+0000 7ff9ae000640 1 -- 172.21.15.131:0/2868642207 >> v2:172.21.15.157:3300/0 conn(0x7ff9b0070fb0 msgr2=0x7ff9b015ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:11.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.705+0000 7ff9ae000640 1 --2- 172.21.15.131:0/2868642207 >> v2:172.21.15.157:3300/0 conn(0x7ff9b0070fb0 0x7ff9b015ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.705+0000 7ff9ae000640 1 -- 172.21.15.131:0/2868642207 >> v1:172.21.15.131:6789/0 conn(0x7ff9b00cefb0 legacy=0x7ff9b00d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.705+0000 7ff9ae000640 1 -- 172.21.15.131:0/2868642207 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b00d1120 con 0x7ff9b0070300 2024-04-28T12:39:11.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.705+0000 7ff9ad600640 1 -- 172.21.15.131:0/2868642207 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4003405887 0 0) 0x7ff9a0001110 con 0x7ff9b00cefb0 2024-04-28T12:39:11.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.706+0000 7ff9ae000640 1 --2- 172.21.15.131:0/2868642207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9b0070300 0x7ff9b0070a70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00b22f0 tx=0x7ff99802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a941f3cdadf3cf server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:11.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.706+0000 7ff9ad600640 1 -- 172.21.15.131:0/2868642207 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998037070 con 0x7ff9b0070300 2024-04-28T12:39:11.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.706+0000 7ff9ad600640 1 -- 172.21.15.131:0/2868642207 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff998032440 con 0x7ff9b0070300 2024-04-28T12:39:11.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.706+0000 7ff9ad600640 1 -- 172.21.15.131:0/2868642207 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99802ae60 con 0x7ff9b0070300 2024-04-28T12:39:11.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.707+0000 7ff9b5e00640 1 -- 172.21.15.131:0/2868642207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9b0070300 msgr2=0x7ff9b0070a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.707+0000 7ff9b5e00640 1 --2- 172.21.15.131:0/2868642207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9b0070300 0x7ff9b0070a70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00b22f0 tx=0x7ff99802a110 comp rx=0 tx=0).stop 2024-04-28T12:39:11.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.707+0000 7ff9b5e00640 1 -- 172.21.15.131:0/2868642207 shutdown_connections 2024-04-28T12:39:11.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.707+0000 7ff9b5e00640 1 --2- 172.21.15.131:0/2868642207 >> v2:172.21.15.157:3300/0 conn(0x7ff9b0070fb0 0x7ff9b015ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.707+0000 7ff9b5e00640 1 --2- 172.21.15.131:0/2868642207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9b0070300 0x7ff9b0070a70 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.707+0000 7ff9b5e00640 1 -- 172.21.15.131:0/2868642207 >> 172.21.15.131:0/2868642207 conn(0x7ff9b00d2db0 msgr2=0x7ff9b00d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:11.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.707+0000 7ff9b5e00640 1 -- 172.21.15.131:0/2868642207 shutdown_connections 2024-04-28T12:39:11.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.707+0000 7ff9b5e00640 1 -- 172.21.15.131:0/2868642207 wait complete. 2024-04-28T12:39:11.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.709+0000 7ff9b5e00640 1 Processor -- start 2024-04-28T12:39:11.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.709+0000 7ff9b5e00640 1 -- start start 2024-04-28T12:39:11.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.710+0000 7ff9b5e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9b0070300 0x7ff9b00cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.710+0000 7ff9b5e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9b0070fb0 0x7ff9b0155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.710+0000 7ff9b5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9b00cefb0 0x7ff9b0155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.710+0000 7ff9b5e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b00d19c0 con 0x7ff9b00cefb0 2024-04-28T12:39:11.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.710+0000 7ff9b5e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b00d16e0 con 0x7ff9b0070300 2024-04-28T12:39:11.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.710+0000 7ff9b5e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b00d1ca0 con 0x7ff9b0070fb0 2024-04-28T12:39:11.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.710+0000 7ff9af400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9b00cefb0 0x7ff9b0155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.710+0000 7ff9af400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9b00cefb0 0x7ff9b0155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57248/0 (socket says 172.21.15.131:57248) 2024-04-28T12:39:11.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.710+0000 7ff9af400640 1 -- 172.21.15.131:0/1655382748 learned_addr learned my addr 172.21.15.131:0/1655382748 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:11.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.710+0000 7ff9aea00640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9b0070300 0x7ff9b00cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9ae000640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9b0070fb0 0x7ff9b0155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9af400640 1 -- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9b0070fb0 msgr2=0x7ff9b0155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9af400640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9b0070fb0 0x7ff9b0155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9af400640 1 -- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9b0070300 msgr2=0x7ff9b00cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9af400640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9b0070300 0x7ff9b00cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9af400640 1 -- 172.21.15.131:0/1655382748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b0155fd0 con 0x7ff9b00cefb0 2024-04-28T12:39:11.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9aea00640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9b0070300 0x7ff9b00cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9af400640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9b00cefb0 0x7ff9b0155200 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8007e40 tx=0x7ff9a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9a7400640 1 -- 172.21.15.131:0/1655382748 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8014620 con 0x7ff9b00cefb0 2024-04-28T12:39:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9a7400640 1 -- 172.21.15.131:0/1655382748 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a8019070 con 0x7ff9b00cefb0 2024-04-28T12:39:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9a7400640 1 -- 172.21.15.131:0/1655382748 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8014a20 con 0x7ff9b00cefb0 2024-04-28T12:39:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9b5e00640 1 -- 172.21.15.131:0/1655382748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b00ca420 con 0x7ff9b00cefb0 2024-04-28T12:39:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.711+0000 7ff9b5e00640 1 -- 172.21.15.131:0/1655382748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b00ca9e0 con 0x7ff9b00cefb0 2024-04-28T12:39:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.713+0000 7ff9b5e00640 1 -- 172.21.15.131:0/1655382748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b0069350 con 0x7ff9b00cefb0 2024-04-28T12:39:11.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.714+0000 7ff9a7400640 1 -- 172.21.15.131:0/1655382748 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9a8014bc0 con 0x7ff9b00cefb0 2024-04-28T12:39:11.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.714+0000 7ff9a7400640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff98c06c8f0 0x7ff98c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:11.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.715+0000 7ff9a7400640 1 -- 172.21.15.131:0/1655382748 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7ff9a808c690 con 0x7ff9b00cefb0 2024-04-28T12:39:11.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.715+0000 7ff9aea00640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff98c06c8f0 0x7ff98c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:11.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.716+0000 7ff9aea00640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff98c06c8f0 0x7ff98c06edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00013b0 tx=0x7ff9a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:11.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.719+0000 7ff9a7400640 1 -- 172.21.15.131:0/1655382748 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff9a805a890 con 0x7ff9b00cefb0 2024-04-28T12:39:11.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.921+0000 7ff9b5e00640 1 -- 172.21.15.131:0/1655382748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9b0068d80 con 0x7ff9b00cefb0 2024-04-28T12:39:11.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.922+0000 7ff9a7400640 1 -- 172.21.15.131:0/1655382748 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16979 (secure 0 0 0) 0x7ff9a805a190 con 0x7ff9b00cefb0 2024-04-28T12:39:11.919 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:11.920 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":37,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:39:10.533721+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:39:08.310817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-28T12:39:09.505586+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:39:11.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.925+0000 7ff9b5e00640 1 -- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff98c06c8f0 msgr2=0x7ff98c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.925+0000 7ff9b5e00640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff98c06c8f0 0x7ff98c06edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00013b0 tx=0x7ff9a0009040 comp rx=0 tx=0).stop 2024-04-28T12:39:11.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.925+0000 7ff9b5e00640 1 -- 172.21.15.131:0/1655382748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9b00cefb0 msgr2=0x7ff9b0155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:11.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.925+0000 7ff9b5e00640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9b00cefb0 0x7ff9b0155200 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8007e40 tx=0x7ff9a800e4b0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.925+0000 7ff9b5e00640 1 -- 172.21.15.131:0/1655382748 shutdown_connections 2024-04-28T12:39:11.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.925+0000 7ff9b5e00640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff98c06c8f0 0x7ff98c06edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.925+0000 7ff9b5e00640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9b00cefb0 0x7ff9b0155200 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.925+0000 7ff9b5e00640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9b0070fb0 0x7ff9b0155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.925+0000 7ff9b5e00640 1 --2- 172.21.15.131:0/1655382748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9b0070300 0x7ff9b00cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:11.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.925+0000 7ff9b5e00640 1 -- 172.21.15.131:0/1655382748 >> 172.21.15.131:0/1655382748 conn(0x7ff9b00d2db0 msgr2=0x7ff9b0162210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:11.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.925+0000 7ff9b5e00640 1 -- 172.21.15.131:0/1655382748 shutdown_connections 2024-04-28T12:39:11.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:11.926+0000 7ff9b5e00640 1 -- 172.21.15.131:0/1655382748 wait complete. 2024-04-28T12:39:11.941 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-28T12:39:12.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.149+0000 7f3504c00640 1 Processor -- start 2024-04-28T12:39:12.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.149+0000 7f3504c00640 1 -- start start 2024-04-28T12:39:12.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.151+0000 7f3504c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f35000cf440 0x7f35000b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:12.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.151+0000 7f3504c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350015d5f0 0x7f350015da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:12.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.151+0000 7f3504c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f350006eaf0 con 0x7f35000cf820 2024-04-28T12:39:12.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.151+0000 7f3504c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f350006f420 con 0x7f35000cf440 2024-04-28T12:39:12.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.151+0000 7f3504c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f350006edd0 con 0x7f350015d5f0 2024-04-28T12:39:12.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.151+0000 7f34fde00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f35000cf440 0x7f35000b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:12.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.151+0000 7f34fde00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f35000cf440 0x7f35000b5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57258/0 (socket says 172.21.15.131:57258) 2024-04-28T12:39:12.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.151+0000 7f34fde00640 1 -- 172.21.15.131:0/1218130414 learned_addr learned my addr 172.21.15.131:0/1218130414 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:12.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.151+0000 7f34fd400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f35000cf820 0x7f35000cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38722/0 (socket says 172.21.15.131:38722) 2024-04-28T12:39:12.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.151+0000 7f34fe800640 1 --2- 172.21.15.131:0/1218130414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350015d5f0 0x7f350015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:12.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.152+0000 7f34fde00640 1 -- 172.21.15.131:0/1218130414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350015d5f0 msgr2=0x7f350015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:12.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.152+0000 7f34fde00640 1 --2- 172.21.15.131:0/1218130414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350015d5f0 0x7f350015da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:12.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.152+0000 7f34fde00640 1 -- 172.21.15.131:0/1218130414 >> v1:172.21.15.131:6789/0 conn(0x7f35000cf820 legacy=0x7f35000cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:12.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.152+0000 7f34fde00640 1 -- 172.21.15.131:0/1218130414 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350006f0b0 con 0x7f35000cf440 2024-04-28T12:39:12.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.152+0000 7f34fca00640 1 -- 172.21.15.131:0/1218130414 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 834017048 0 0) 0x7f350006eaf0 con 0x7f35000cf820 2024-04-28T12:39:12.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.152+0000 7f34fe800640 1 --2- 172.21.15.131:0/1218130414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350015d5f0 0x7f350015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.153+0000 7f34fde00640 1 --2- 172.21.15.131:0/1218130414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35000cf440 0x7f35000b5250 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009870 tx=0x7f34e802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2644ffcb5ed77c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.153+0000 7f34fca00640 1 -- 172.21.15.131:0/1218130414 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8037070 con 0x7f35000cf440 2024-04-28T12:39:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.153+0000 7f34fca00640 1 -- 172.21.15.131:0/1218130414 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e8032440 con 0x7f35000cf440 2024-04-28T12:39:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.153+0000 7f34fca00640 1 -- 172.21.15.131:0/1218130414 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8032810 con 0x7f35000cf440 2024-04-28T12:39:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.154+0000 7f3504c00640 1 -- 172.21.15.131:0/1218130414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35000cf440 msgr2=0x7f35000b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.154+0000 7f3504c00640 1 --2- 172.21.15.131:0/1218130414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35000cf440 0x7f35000b5250 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009870 tx=0x7f34e802a1c0 comp rx=0 tx=0).stop 2024-04-28T12:39:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.154+0000 7f3504c00640 1 -- 172.21.15.131:0/1218130414 shutdown_connections 2024-04-28T12:39:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.154+0000 7f3504c00640 1 --2- 172.21.15.131:0/1218130414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f350015d5f0 0x7f350015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:12.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.154+0000 7f3504c00640 1 --2- 172.21.15.131:0/1218130414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35000cf440 0x7f35000b5250 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:12.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.154+0000 7f3504c00640 1 -- 172.21.15.131:0/1218130414 >> 172.21.15.131:0/1218130414 conn(0x7f35000b2900 msgr2=0x7f35000b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:12.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.154+0000 7f3504c00640 1 -- 172.21.15.131:0/1218130414 shutdown_connections 2024-04-28T12:39:12.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.154+0000 7f3504c00640 1 -- 172.21.15.131:0/1218130414 wait complete. 2024-04-28T12:39:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.155+0000 7f3504c00640 1 Processor -- start 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.156+0000 7f3504c00640 1 -- start start 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.156+0000 7f3504c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35000cf440 0x7f35001f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f3504c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35000cf820 0x7f350013fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f3504c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350015d5f0 0x7f350015a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f3504c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f35000d0390 con 0x7f35000cf440 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f3504c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f350006f8b0 con 0x7f350015d5f0 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f3504c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f35000d0510 con 0x7f35000cf820 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f34fde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35000cf440 0x7f35001f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f34fde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35000cf440 0x7f35001f3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57268/0 (socket says 172.21.15.131:57268) 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f34fde00640 1 -- 172.21.15.131:0/1459466061 learned_addr learned my addr 172.21.15.131:0/1459466061 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f34fde00640 1 -- 172.21.15.131:0/1459466061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35000cf820 msgr2=0x7f350013fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f34fde00640 1 --2- 172.21.15.131:0/1459466061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35000cf820 0x7f350013fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f34fde00640 1 -- 172.21.15.131:0/1459466061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350015d5f0 msgr2=0x7f350015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f34fde00640 1 --2- 172.21.15.131:0/1459466061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350015d5f0 0x7f350015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:12.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.157+0000 7f34fde00640 1 -- 172.21.15.131:0/1459466061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350013dc80 con 0x7f35000cf440 2024-04-28T12:39:12.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.158+0000 7f34fde00640 1 --2- 172.21.15.131:0/1459466061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35000cf440 0x7f35001f3440 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f34e80097c0 tx=0x7f34e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:12.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.158+0000 7f34f6000640 1 -- 172.21.15.131:0/1459466061 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8041070 con 0x7f35000cf440 2024-04-28T12:39:12.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.158+0000 7f34f6000640 1 -- 172.21.15.131:0/1459466061 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e8004220 con 0x7f35000cf440 2024-04-28T12:39:12.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.158+0000 7f34f6000640 1 -- 172.21.15.131:0/1459466061 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8037040 con 0x7f35000cf440 2024-04-28T12:39:12.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.158+0000 7f3504c00640 1 -- 172.21.15.131:0/1459466061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350013df60 con 0x7f35000cf440 2024-04-28T12:39:12.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.158+0000 7f3504c00640 1 -- 172.21.15.131:0/1459466061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350013e520 con 0x7f35000cf440 2024-04-28T12:39:12.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.159+0000 7f3504c00640 1 -- 172.21.15.131:0/1459466061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3500069350 con 0x7f35000cf440 2024-04-28T12:39:12.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.160+0000 7f34f6000640 1 -- 172.21.15.131:0/1459466061 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34e802ad10 con 0x7f35000cf440 2024-04-28T12:39:12.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.161+0000 7f34f6000640 1 --2- 172.21.15.131:0/1459466061 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f34dc06c8f0 0x7f34dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:12.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.161+0000 7f34f6000640 1 -- 172.21.15.131:0/1459466061 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f34e80ac320 con 0x7f35000cf440 2024-04-28T12:39:12.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.162+0000 7f34fd400640 1 --2- 172.21.15.131:0/1459466061 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f34dc06c8f0 0x7f34dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:12.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.163+0000 7f34fd400640 1 --2- 172.21.15.131:0/1459466061 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f34dc06c8f0 0x7f34dc06edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f34f0007fd0 tx=0x7f34f0007920 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.166+0000 7f34f6000640 1 -- 172.21.15.131:0/1459466061 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34e807a650 con 0x7f35000cf440 2024-04-28T12:39:12.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:12.402+0000 7f3504c00640 1 -- 172.21.15.131:0/1459466061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3500145d60 con 0x7f35000cf440 2024-04-28T12:39:13.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.111+0000 7f34f6000640 1 -- 172.21.15.131:0/1459466061 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f34e8079f50 con 0x7f35000cf440 2024-04-28T12:39:13.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.114+0000 7f3504c00640 1 -- 172.21.15.131:0/1459466061 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f34dc06c8f0 msgr2=0x7f34dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:13.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.114+0000 7f3504c00640 1 --2- 172.21.15.131:0/1459466061 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f34dc06c8f0 0x7f34dc06edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f34f0007fd0 tx=0x7f34f0007920 comp rx=0 tx=0).stop 2024-04-28T12:39:13.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.114+0000 7f3504c00640 1 -- 172.21.15.131:0/1459466061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35000cf440 msgr2=0x7f35001f3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:13.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.114+0000 7f3504c00640 1 --2- 172.21.15.131:0/1459466061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35000cf440 0x7f35001f3440 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f34e80097c0 tx=0x7f34e80046c0 comp rx=0 tx=0).stop 2024-04-28T12:39:13.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.114+0000 7f3504c00640 1 -- 172.21.15.131:0/1459466061 shutdown_connections 2024-04-28T12:39:13.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.114+0000 7f3504c00640 1 --2- 172.21.15.131:0/1459466061 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f34dc06c8f0 0x7f34dc06edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:13.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.114+0000 7f3504c00640 1 --2- 172.21.15.131:0/1459466061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f350015d5f0 0x7f350015a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:13.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.114+0000 7f3504c00640 1 --2- 172.21.15.131:0/1459466061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35000cf820 0x7f350013fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:13.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.114+0000 7f3504c00640 1 --2- 172.21.15.131:0/1459466061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35000cf440 0x7f35001f3440 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:13.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.114+0000 7f3504c00640 1 -- 172.21.15.131:0/1459466061 >> 172.21.15.131:0/1459466061 conn(0x7f35000b2900 msgr2=0x7f3500144070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:13.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.115+0000 7f3504c00640 1 -- 172.21.15.131:0/1459466061 shutdown_connections 2024-04-28T12:39:13.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.115+0000 7f3504c00640 1 -- 172.21.15.131:0/1459466061 wait complete. 2024-04-28T12:39:13.134 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-28T12:39:13.134 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-28T12:39:13.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.335+0000 7f0a8f200640 1 Processor -- start 2024-04-28T12:39:13.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.335+0000 7f0a8f200640 1 -- start start 2024-04-28T12:39:13.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.337+0000 7f0a8f200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0a9015a7e0 0x7f0a900b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:13.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.337+0000 7f0a8f200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f0a9015d9c0 0x7f0a900b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:13.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.337+0000 7f0a8f200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a900b3900 con 0x7f0a900b5410 2024-04-28T12:39:13.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.337+0000 7f0a8f200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0a9015abc0 con 0x7f0a9015a7e0 2024-04-28T12:39:13.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.337+0000 7f0a8f200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f0a900b3b70 con 0x7f0a9015d9c0 2024-04-28T12:39:13.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.337+0000 7f0a8de00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0a9015a7e0 0x7f0a900b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:13.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.337+0000 7f0a8de00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0a9015a7e0 0x7f0a900b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57274/0 (socket says 172.21.15.131:57274) 2024-04-28T12:39:13.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.337+0000 7f0a8de00640 1 -- 172.21.15.131:0/1646314351 learned_addr learned my addr 172.21.15.131:0/1646314351 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:13.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.337+0000 7f0a8de00640 1 -- 172.21.15.131:0/1646314351 >> v2:172.21.15.157:3300/0 conn(0x7f0a9015d9c0 msgr2=0x7f0a900b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:13.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.337+0000 7f0a8de00640 1 --2- 172.21.15.131:0/1646314351 >> v2:172.21.15.157:3300/0 conn(0x7f0a9015d9c0 0x7f0a900b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:13.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.337+0000 7f0a8de00640 1 -- 172.21.15.131:0/1646314351 >> v1:172.21.15.157:6789/0 conn(0x7f0a900b5410 legacy=0x7f0a9015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:13.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.337+0000 7f0a8de00640 1 -- 172.21.15.131:0/1646314351 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a900b3e50 con 0x7f0a9015a7e0 2024-04-28T12:39:13.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.338+0000 7f0a8de00640 1 --2- 172.21.15.131:0/1646314351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a9015a7e0 0x7f0a900b4650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0a900cd1c0 tx=0x7f0a8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91283e220d6fbc44 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:13.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.339+0000 7f0a8ca00640 1 -- 172.21.15.131:0/1646314351 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80037070 con 0x7f0a9015a7e0 2024-04-28T12:39:13.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.339+0000 7f0a8ca00640 1 -- 172.21.15.131:0/1646314351 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a80032440 con 0x7f0a9015a7e0 2024-04-28T12:39:13.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.339+0000 7f0a8ca00640 1 -- 172.21.15.131:0/1646314351 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80032810 con 0x7f0a9015a7e0 2024-04-28T12:39:13.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.339+0000 7f0a8f200640 1 -- 172.21.15.131:0/1646314351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a9015a7e0 msgr2=0x7f0a900b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:13.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.339+0000 7f0a8f200640 1 --2- 172.21.15.131:0/1646314351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a9015a7e0 0x7f0a900b4650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0a900cd1c0 tx=0x7f0a8002a230 comp rx=0 tx=0).stop 2024-04-28T12:39:13.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.339+0000 7f0a8f200640 1 -- 172.21.15.131:0/1646314351 shutdown_connections 2024-04-28T12:39:13.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.339+0000 7f0a8f200640 1 --2- 172.21.15.131:0/1646314351 >> v2:172.21.15.157:3300/0 conn(0x7f0a9015d9c0 0x7f0a900b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:13.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.339+0000 7f0a8f200640 1 --2- 172.21.15.131:0/1646314351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a9015a7e0 0x7f0a900b4650 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:13.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.339+0000 7f0a8f200640 1 -- 172.21.15.131:0/1646314351 >> 172.21.15.131:0/1646314351 conn(0x7f0a90068090 msgr2=0x7f0a900684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:13.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.339+0000 7f0a8f200640 1 -- 172.21.15.131:0/1646314351 shutdown_connections 2024-04-28T12:39:13.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.339+0000 7f0a8f200640 1 -- 172.21.15.131:0/1646314351 wait complete. 2024-04-28T12:39:13.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.341+0000 7f0a8f200640 1 Processor -- start 2024-04-28T12:39:13.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.341+0000 7f0a8f200640 1 -- start start 2024-04-28T12:39:13.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.342+0000 7f0a8f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a900b5410 0x7f0a901f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:13.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.342+0000 7f0a8f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a9015a7e0 0x7f0a90152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:13.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.342+0000 7f0a8f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a9015d9c0 0x7f0a90146970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:13.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.342+0000 7f0a8f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0a900b4b90 con 0x7f0a900b5410 2024-04-28T12:39:13.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.342+0000 7f0a8f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9015e010 con 0x7f0a9015d9c0 2024-04-28T12:39:13.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.342+0000 7f0a8f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9015e2e0 con 0x7f0a9015a7e0 2024-04-28T12:39:13.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.342+0000 7f0a8de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a900b5410 0x7f0a901f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:13.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.342+0000 7f0a8de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a900b5410 0x7f0a901f5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57284/0 (socket says 172.21.15.131:57284) 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.342+0000 7f0a8de00640 1 -- 172.21.15.131:0/4007326260 learned_addr learned my addr 172.21.15.131:0/4007326260 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.343+0000 7f0a8de00640 1 -- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a9015a7e0 msgr2=0x7f0a90152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.343+0000 7f0a8d400640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a9015a7e0 0x7f0a90152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.343+0000 7f0a8de00640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a9015a7e0 0x7f0a90152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.343+0000 7f0a8e800640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a9015d9c0 0x7f0a90146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.343+0000 7f0a8de00640 1 -- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a9015d9c0 msgr2=0x7f0a90146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.343+0000 7f0a8de00640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a9015d9c0 0x7f0a90146970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.343+0000 7f0a8de00640 1 -- 172.21.15.131:0/4007326260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a90152e30 con 0x7f0a900b5410 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.343+0000 7f0a8de00640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a900b5410 0x7f0a901f5ab0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c00 tx=0x7f0a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.343+0000 7f0a86000640 1 -- 172.21.15.131:0/4007326260 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80041070 con 0x7f0a900b5410 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.343+0000 7f0a86000640 1 -- 172.21.15.131:0/4007326260 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8003c870 con 0x7f0a900b5410 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.344+0000 7f0a8f200640 1 -- 172.21.15.131:0/4007326260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a90153110 con 0x7f0a900b5410 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.344+0000 7f0a86000640 1 -- 172.21.15.131:0/4007326260 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80037040 con 0x7f0a900b5410 2024-04-28T12:39:13.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.344+0000 7f0a8f200640 1 -- 172.21.15.131:0/4007326260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a901536d0 con 0x7f0a900b5410 2024-04-28T12:39:13.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.345+0000 7f0a8f200640 1 -- 172.21.15.131:0/4007326260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a900506c0 con 0x7f0a900b5410 2024-04-28T12:39:13.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.346+0000 7f0a86000640 1 -- 172.21.15.131:0/4007326260 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a80044070 con 0x7f0a900b5410 2024-04-28T12:39:13.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.346+0000 7f0a86000640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0a7006c8f0 0x7f0a7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:13.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.347+0000 7f0a86000640 1 -- 172.21.15.131:0/4007326260 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0a800ac930 con 0x7f0a900b5410 2024-04-28T12:39:13.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.347+0000 7f0a8d400640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0a7006c8f0 0x7f0a7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:13.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.348+0000 7f0a8d400640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0a7006c8f0 0x7f0a7006edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0a9015be00 tx=0x7f0a7c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:13.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.351+0000 7f0a86000640 1 -- 172.21.15.131:0/4007326260 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a8007ad20 con 0x7f0a900b5410 2024-04-28T12:39:13.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:13.593+0000 7f0a8f200640 1 -- 172.21.15.131:0/4007326260 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0a900b33c0 con 0x7f0a900b5410 2024-04-28T12:39:14.039 INFO:tasks.ceph.mds.d.smithi157.stderr:2024-04-28T12:39:14.042+0000 7f1dffc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-28T12:39:14.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.114+0000 7f0a86000640 1 -- 172.21.15.131:0/4007326260 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f0a8007a620 con 0x7f0a900b5410 2024-04-28T12:39:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.118+0000 7f0a8f200640 1 -- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0a7006c8f0 msgr2=0x7f0a7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.118+0000 7f0a8f200640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0a7006c8f0 0x7f0a7006edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0a9015be00 tx=0x7f0a7c009040 comp rx=0 tx=0).stop 2024-04-28T12:39:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.118+0000 7f0a8f200640 1 -- 172.21.15.131:0/4007326260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a900b5410 msgr2=0x7f0a901f5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.118+0000 7f0a8f200640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a900b5410 0x7f0a901f5ab0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c00 tx=0x7f0a800046c0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.118+0000 7f0a8f200640 1 -- 172.21.15.131:0/4007326260 shutdown_connections 2024-04-28T12:39:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.118+0000 7f0a8f200640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f0a7006c8f0 0x7f0a7006edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.118+0000 7f0a8f200640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a9015d9c0 0x7f0a90146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.118+0000 7f0a8f200640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a9015a7e0 0x7f0a90152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.118+0000 7f0a8f200640 1 --2- 172.21.15.131:0/4007326260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a900b5410 0x7f0a901f5ab0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.118+0000 7f0a8f200640 1 -- 172.21.15.131:0/4007326260 >> 172.21.15.131:0/4007326260 conn(0x7f0a90068090 msgr2=0x7f0a900cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.118+0000 7f0a8f200640 1 -- 172.21.15.131:0/4007326260 shutdown_connections 2024-04-28T12:39:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.118+0000 7f0a8f200640 1 -- 172.21.15.131:0/4007326260 wait complete. 2024-04-28T12:39:14.136 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-28T12:39:14.136 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:39:14.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.342+0000 7f8f6e600640 1 Processor -- start 2024-04-28T12:39:14.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.342+0000 7f8f6e600640 1 -- start start 2024-04-28T12:39:14.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.344+0000 7f8f6e600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8f68145330 0x7f8f680b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.344+0000 7f8f6e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f681628e0 0x7f8f68162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.344+0000 7f8f6e600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f68145710 con 0x7f8f680af720 2024-04-28T12:39:14.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.344+0000 7f8f6e600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8f68163280 con 0x7f8f68145330 2024-04-28T12:39:14.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.344+0000 7f8f6e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f68073dc0 con 0x7f8f681628e0 2024-04-28T12:39:14.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.344+0000 7f8f6dc00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8f680af720 0x7f8f680afbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38738/0 (socket says 172.21.15.131:38738) 2024-04-28T12:39:14.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.344+0000 7f8f6dc00640 1 -- 172.21.15.131:0/2373256783 learned_addr learned my addr 172.21.15.131:0/2373256783 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:14.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f67e00640 1 --2- 172.21.15.131:0/2373256783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f681628e0 0x7f8f68162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:14.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f6d200640 1 --2- 172.21.15.131:0/2373256783 >> v2:172.21.15.157:3300/0 conn(0x7f8f68145330 0x7f8f680b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:14.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375008730 0 0) 0x7f8f68145710 con 0x7f8f680af720 2024-04-28T12:39:14.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f50003520 con 0x7f8f680af720 2024-04-28T12:39:14.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3888110062 0 0) 0x7f8f50003520 con 0x7f8f680af720 2024-04-28T12:39:14.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f681628e0 msgr2=0x7f8f68162d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:14.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f67400640 1 --2- 172.21.15.131:0/2373256783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f681628e0 0x7f8f68162d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 >> v2:172.21.15.157:3300/0 conn(0x7f8f68145330 msgr2=0x7f8f680b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f67400640 1 --2- 172.21.15.131:0/2373256783 >> v2:172.21.15.157:3300/0 conn(0x7f8f68145330 0x7f8f680b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f680ae960 con 0x7f8f680af720 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f6d200640 1 --2- 172.21.15.131:0/2373256783 >> v2:172.21.15.157:3300/0 conn(0x7f8f68145330 0x7f8f680b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f8f600019f0 con 0x7f8f680af720 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 >> v1:172.21.15.131:6789/0 conn(0x7f8f680af720 legacy=0x7f8f680afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.345+0000 7f8f67400640 1 --2- 172.21.15.131:0/2373256783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f681628e0 0x7f8f500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.346+0000 7f8f67400640 1 --2- 172.21.15.131:0/2373256783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f50003f30 0x7f8f50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.346+0000 7f8f67400640 1 --2- 172.21.15.131:0/2373256783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f50004d40 0x7f8f500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.346+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8f68073dc0 con 0x7f8f50003f30 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.346+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5000b9c0 con 0x7f8f50004d40 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.346+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5000bb90 con 0x7f8f681628e0 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.346+0000 7f8f67e00640 1 --2- 172.21.15.131:0/2373256783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f50003f30 0x7f8f50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.346+0000 7f8f67e00640 1 -- 172.21.15.131:0/2373256783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f681628e0 msgr2=0x7f8f500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.346+0000 7f8f67e00640 1 --2- 172.21.15.131:0/2373256783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f681628e0 0x7f8f500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.346+0000 7f8f67e00640 1 -- 172.21.15.131:0/2373256783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f50004d40 msgr2=0x7f8f500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:14.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.346+0000 7f8f67e00640 1 --2- 172.21.15.131:0/2373256783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f50004d40 0x7f8f500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.346+0000 7f8f67e00640 1 -- 172.21.15.131:0/2373256783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f50003520 con 0x7f8f50003f30 2024-04-28T12:39:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.347+0000 7f8f67e00640 1 --2- 172.21.15.131:0/2373256783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f50003f30 0x7f8f50004800 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009ac0 tx=0x7f8f50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98304a06b40e77f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.347+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54036070 con 0x7f8f50003f30 2024-04-28T12:39:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.348+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f54031400 con 0x7f8f50003f30 2024-04-28T12:39:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.348+0000 7f8f67400640 1 -- 172.21.15.131:0/2373256783 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54009690 con 0x7f8f50003f30 2024-04-28T12:39:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.348+0000 7f8f6e600640 1 -- 172.21.15.131:0/2373256783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f50003f30 msgr2=0x7f8f50004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.348+0000 7f8f6e600640 1 --2- 172.21.15.131:0/2373256783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f50003f30 0x7f8f50004800 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009ac0 tx=0x7f8f50005830 comp rx=0 tx=0).stop 2024-04-28T12:39:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.348+0000 7f8f6dc00640 1 -- 172.21.15.131:0/2373256783 reap_dead start 2024-04-28T12:39:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.348+0000 7f8f6e600640 1 -- 172.21.15.131:0/2373256783 shutdown_connections 2024-04-28T12:39:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.348+0000 7f8f6e600640 1 -- 172.21.15.131:0/2373256783 >> 172.21.15.131:0/2373256783 conn(0x7f8f68068090 msgr2=0x7f8f68068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.348+0000 7f8f6e600640 1 -- 172.21.15.131:0/2373256783 shutdown_connections 2024-04-28T12:39:14.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.348+0000 7f8f6e600640 1 -- 172.21.15.131:0/2373256783 wait complete. 2024-04-28T12:39:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.350+0000 7f8f6e600640 1 Processor -- start 2024-04-28T12:39:14.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.350+0000 7f8f6e600640 1 -- start start 2024-04-28T12:39:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.351+0000 7f8f6e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f50003f30 0x7f8f681f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.351+0000 7f8f6e600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f680b2140 0x7f8f6815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.351+0000 7f8f6e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f680c8f00 0x7f8f680c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.351+0000 7f8f6e600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680af200 con 0x7f8f680b2140 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.351+0000 7f8f6e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680aef20 con 0x7f8f680c8f00 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.351+0000 7f8f6e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8f680b3e80 con 0x7f8f50003f30 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.351+0000 7f8f67e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f680b2140 0x7f8f6815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.351+0000 7f8f67e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f680b2140 0x7f8f6815c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57310/0 (socket says 172.21.15.131:57310) 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.352+0000 7f8f67e00640 1 -- 172.21.15.131:0/4087621502 learned_addr learned my addr 172.21.15.131:0/4087621502 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.352+0000 7f8f6d200640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f50003f30 0x7f8f681f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.352+0000 7f8f67e00640 1 -- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f50003f30 msgr2=0x7f8f681f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.352+0000 7f8f67e00640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f50003f30 0x7f8f681f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.352+0000 7f8f6dc00640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f680c8f00 0x7f8f680c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.352+0000 7f8f67e00640 1 -- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f680c8f00 msgr2=0x7f8f680c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.352+0000 7f8f67e00640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f680c8f00 0x7f8f680c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.352+0000 7f8f67e00640 1 -- 172.21.15.131:0/4087621502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f680b25c0 con 0x7f8f680b2140 2024-04-28T12:39:14.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.352+0000 7f8f6dc00640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f680c8f00 0x7f8f680c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.352+0000 7f8f67e00640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f680b2140 0x7f8f6815c0f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.353+0000 7f8f64c00640 1 -- 172.21.15.131:0/4087621502 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54040020 con 0x7f8f680b2140 2024-04-28T12:39:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.353+0000 7f8f64c00640 1 -- 172.21.15.131:0/4087621502 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f54003c40 con 0x7f8f680b2140 2024-04-28T12:39:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.353+0000 7f8f64c00640 1 -- 172.21.15.131:0/4087621502 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54036040 con 0x7f8f680b2140 2024-04-28T12:39:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.353+0000 7f8f6e600640 1 -- 172.21.15.131:0/4087621502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f680b2840 con 0x7f8f680b2140 2024-04-28T12:39:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.353+0000 7f8f6e600640 1 -- 172.21.15.131:0/4087621502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f680b2e00 con 0x7f8f680b2140 2024-04-28T12:39:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.354+0000 7f8f6e600640 1 -- 172.21.15.131:0/4087621502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f68069350 con 0x7f8f680b2140 2024-04-28T12:39:14.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.355+0000 7f8f64c00640 1 -- 172.21.15.131:0/4087621502 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f54043020 con 0x7f8f680b2140 2024-04-28T12:39:14.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.356+0000 7f8f64c00640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8f4806c8e0 0x7f8f4806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.356+0000 7f8f64c00640 1 -- 172.21.15.131:0/4087621502 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f8f540abe30 con 0x7f8f680b2140 2024-04-28T12:39:14.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.357+0000 7f8f6d200640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8f4806c8e0 0x7f8f4806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:14.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.358+0000 7f8f6d200640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8f4806c8e0 0x7f8f4806eda0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c006a90 tx=0x7f8f5c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.361+0000 7f8f64c00640 1 -- 172.21.15.131:0/4087621502 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f5404a070 con 0x7f8f680b2140 2024-04-28T12:39:14.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.600+0000 7f8f6e600640 1 -- 172.21.15.131:0/4087621502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f68075250 con 0x7f8f680b2140 2024-04-28T12:39:14.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.601+0000 7f8f64c00640 1 -- 172.21.15.131:0/4087621502 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5189 (secure 0 0 0) 0x7f8f680b25c0 con 0x7f8f680b2140 2024-04-28T12:39:14.598 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:14.598 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/361318591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":361318591},{"type":"v1","addr":"172.21.15.157:6836","nonce":361318591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/102335005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":102335005},{"type":"v1","addr":"172.21.15.131:6835","nonce":102335005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1609059776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1609059776},{"type":"v1","addr":"172.21.15.131:6833","nonce":1609059776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:39:10.533686+0000","modified":"2024-04-28T12:39:13.595067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34610":{"gid":34610,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/837933884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":837933884},{"type":"v1","addr":"172.21.15.157:6837","nonce":837933884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34610,"qdb_cluster":[34610]},"id":2}]} 2024-04-28T12:39:14.598 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2024-04-28T12:39:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.604+0000 7f8f6e600640 1 -- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8f4806c8e0 msgr2=0x7f8f4806eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.604+0000 7f8f6e600640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8f4806c8e0 0x7f8f4806eda0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c006a90 tx=0x7f8f5c008040 comp rx=0 tx=0).stop 2024-04-28T12:39:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.604+0000 7f8f6e600640 1 -- 172.21.15.131:0/4087621502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f680b2140 msgr2=0x7f8f6815c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.604+0000 7f8f6e600640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f680b2140 0x7f8f6815c0f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f54004730 comp rx=0 tx=0).stop 2024-04-28T12:39:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.604+0000 7f8f6e600640 1 -- 172.21.15.131:0/4087621502 shutdown_connections 2024-04-28T12:39:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.604+0000 7f8f6e600640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8f4806c8e0 0x7f8f4806eda0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.604+0000 7f8f6e600640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8f680c8f00 0x7f8f680c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.604+0000 7f8f6e600640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f680b2140 0x7f8f6815c0f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.604+0000 7f8f6e600640 1 --2- 172.21.15.131:0/4087621502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8f50003f30 0x7f8f681f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.604+0000 7f8f6e600640 1 -- 172.21.15.131:0/4087621502 >> 172.21.15.131:0/4087621502 conn(0x7f8f68068090 msgr2=0x7f8f68144280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:14.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.605+0000 7f8f6e600640 1 -- 172.21.15.131:0/4087621502 shutdown_connections 2024-04-28T12:39:14.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.605+0000 7f8f6e600640 1 -- 172.21.15.131:0/4087621502 wait complete. 2024-04-28T12:39:14.619 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:39:14.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.823+0000 7fe9bb200640 1 Processor -- start 2024-04-28T12:39:14.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.823+0000 7fe9bb200640 1 -- start start 2024-04-28T12:39:14.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.824+0000 7fe9bb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9bc072620 0x7fe9bc0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.824+0000 7fe9bb200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe9bc0ce2b0 0x7fe9bc06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.824+0000 7fe9bb200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9bc072a80 con 0x7fe9bc0cd180 2024-04-28T12:39:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.824+0000 7fe9bb200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe9bc0cd560 con 0x7fe9bc0ce2b0 2024-04-28T12:39:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.824+0000 7fe9bb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe9bc072d00 con 0x7fe9bc072620 2024-04-28T12:39:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.825+0000 7fe9ba800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe9bc0ce2b0 0x7fe9bc06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.825+0000 7fe9ba800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe9bc0ce2b0 0x7fe9bc06dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57318/0 (socket says 172.21.15.131:57318) 2024-04-28T12:39:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.825+0000 7fe9ba800640 1 -- 172.21.15.131:0/1667970643 learned_addr learned my addr 172.21.15.131:0/1667970643 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.825+0000 7fe9b9400640 1 --2- 172.21.15.131:0/1667970643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9bc072620 0x7fe9bc0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.825+0000 7fe9ba800640 1 -- 172.21.15.131:0/1667970643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9bc072620 msgr2=0x7fe9bc0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.825+0000 7fe9ba800640 1 --2- 172.21.15.131:0/1667970643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9bc072620 0x7fe9bc0cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.825+0000 7fe9ba800640 1 -- 172.21.15.131:0/1667970643 >> v1:172.21.15.157:6789/0 conn(0x7fe9bc0cd180 legacy=0x7fe9bc15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:14.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.825+0000 7fe9ba800640 1 -- 172.21.15.131:0/1667970643 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc072fe0 con 0x7fe9bc0ce2b0 2024-04-28T12:39:14.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.826+0000 7fe9ba800640 1 --2- 172.21.15.131:0/1667970643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9bc0ce2b0 0x7fe9bc06dac0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009870 tx=0x7fe9b402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1c790a462bbd13 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:14.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.826+0000 7fe9b8a00640 1 -- 172.21.15.131:0/1667970643 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4037070 con 0x7fe9bc0ce2b0 2024-04-28T12:39:14.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.827+0000 7fe9b8a00640 1 -- 172.21.15.131:0/1667970643 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b4032440 con 0x7fe9bc0ce2b0 2024-04-28T12:39:14.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.827+0000 7fe9b8a00640 1 -- 172.21.15.131:0/1667970643 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4032810 con 0x7fe9bc0ce2b0 2024-04-28T12:39:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.827+0000 7fe9bb200640 1 -- 172.21.15.131:0/1667970643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9bc0ce2b0 msgr2=0x7fe9bc06dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.827+0000 7fe9bb200640 1 --2- 172.21.15.131:0/1667970643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9bc0ce2b0 0x7fe9bc06dac0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009870 tx=0x7fe9b402a210 comp rx=0 tx=0).stop 2024-04-28T12:39:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.827+0000 7fe9bb200640 1 -- 172.21.15.131:0/1667970643 shutdown_connections 2024-04-28T12:39:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.827+0000 7fe9bb200640 1 --2- 172.21.15.131:0/1667970643 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9bc0ce2b0 0x7fe9bc06dac0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.827+0000 7fe9bb200640 1 --2- 172.21.15.131:0/1667970643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9bc072620 0x7fe9bc0cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.827+0000 7fe9bb200640 1 -- 172.21.15.131:0/1667970643 >> 172.21.15.131:0/1667970643 conn(0x7fe9bc068090 msgr2=0x7fe9bc068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.827+0000 7fe9bb200640 1 -- 172.21.15.131:0/1667970643 shutdown_connections 2024-04-28T12:39:14.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.827+0000 7fe9bb200640 1 -- 172.21.15.131:0/1667970643 wait complete. 2024-04-28T12:39:14.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.829+0000 7fe9bb200640 1 Processor -- start 2024-04-28T12:39:14.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.829+0000 7fe9bb200640 1 -- start start 2024-04-28T12:39:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.830+0000 7fe9bb200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9bc072620 0x7fe9bc13b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.830+0000 7fe9bb200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9bc0cd180 0x7fe9bc0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.830+0000 7fe9bb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9bc0ce2b0 0x7fe9bc0ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.830+0000 7fe9bb200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc073da0 con 0x7fe9bc072620 2024-04-28T12:39:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.830+0000 7fe9bb200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc0735a0 con 0x7fe9bc0cd180 2024-04-28T12:39:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.830+0000 7fe9bb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe9bc06e000 con 0x7fe9bc0ce2b0 2024-04-28T12:39:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.830+0000 7fe9b9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9bc072620 0x7fe9bc13b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.830+0000 7fe9b9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9bc072620 0x7fe9bc13b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57332/0 (socket says 172.21.15.131:57332) 2024-04-28T12:39:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.830+0000 7fe9b9e00640 1 -- 172.21.15.131:0/3349684778 learned_addr learned my addr 172.21.15.131:0/3349684778 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.831+0000 7fe9b9e00640 1 -- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9bc0ce2b0 msgr2=0x7fe9bc0ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:14.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.831+0000 7fe9b9e00640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9bc0ce2b0 0x7fe9bc0ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.831+0000 7fe9b9400640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9bc0cd180 0x7fe9bc0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.831+0000 7fe9b9e00640 1 -- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9bc0cd180 msgr2=0x7fe9bc0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.831+0000 7fe9b9e00640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9bc0cd180 0x7fe9bc0b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.831+0000 7fe9b9e00640 1 -- 172.21.15.131:0/3349684778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc13b8f0 con 0x7fe9bc072620 2024-04-28T12:39:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.831+0000 7fe9b9e00640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9bc072620 0x7fe9bc13b3b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00c460 tx=0x7fe9ac00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.831+0000 7fe9b2000640 1 -- 172.21.15.131:0/3349684778 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac016020 con 0x7fe9bc072620 2024-04-28T12:39:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.831+0000 7fe9b2000640 1 -- 172.21.15.131:0/3349684778 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac012870 con 0x7fe9bc072620 2024-04-28T12:39:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.831+0000 7fe9b2000640 1 -- 172.21.15.131:0/3349684778 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac012c60 con 0x7fe9bc072620 2024-04-28T12:39:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.831+0000 7fe9bb200640 1 -- 172.21.15.131:0/3349684778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9bc0b5a30 con 0x7fe9bc072620 2024-04-28T12:39:14.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.831+0000 7fe9bb200640 1 -- 172.21.15.131:0/3349684778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9bc0b4c70 con 0x7fe9bc072620 2024-04-28T12:39:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.832+0000 7fe9bb200640 1 -- 172.21.15.131:0/3349684778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9bc069350 con 0x7fe9bc072620 2024-04-28T12:39:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.833+0000 7fe9b2000640 1 -- 172.21.15.131:0/3349684778 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9ac019020 con 0x7fe9bc072620 2024-04-28T12:39:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.834+0000 7fe9b2000640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe99806c8f0 0x7fe99806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:14.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.834+0000 7fe9b2000640 1 -- 172.21.15.131:0/3349684778 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe9ac08b710 con 0x7fe9bc072620 2024-04-28T12:39:14.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.835+0000 7fe9b9400640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe99806c8f0 0x7fe99806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:14.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.836+0000 7fe9b9400640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe99806c8f0 0x7fe99806edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0b3cc0 tx=0x7fe9a400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:14.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:14.838+0000 7fe9b2000640 1 -- 172.21.15.131:0/3349684778 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe9ac020070 con 0x7fe9bc072620 2024-04-28T12:39:15.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.038+0000 7fe9bb200640 1 -- 172.21.15.131:0/3349684778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9bc0720e0 con 0x7fe9bc072620 2024-04-28T12:39:15.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.041+0000 7fe9b2000640 1 -- 172.21.15.131:0/3349684778 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16966 (secure 0 0 0) 0x7fe9bc13b8f0 con 0x7fe9bc072620 2024-04-28T12:39:15.038 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:15.038 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":38,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:39:12.083076+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:39:08.310817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-28T12:39:09.505586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:39:15.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.044+0000 7fe9bb200640 1 -- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe99806c8f0 msgr2=0x7fe99806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.044+0000 7fe9bb200640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe99806c8f0 0x7fe99806edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0b3cc0 tx=0x7fe9a400b040 comp rx=0 tx=0).stop 2024-04-28T12:39:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.044+0000 7fe9bb200640 1 -- 172.21.15.131:0/3349684778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9bc072620 msgr2=0x7fe9bc13b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.044+0000 7fe9bb200640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9bc072620 0x7fe9bc13b3b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00c460 tx=0x7fe9ac00c920 comp rx=0 tx=0).stop 2024-04-28T12:39:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.044+0000 7fe9bb200640 1 -- 172.21.15.131:0/3349684778 shutdown_connections 2024-04-28T12:39:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.044+0000 7fe9bb200640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe99806c8f0 0x7fe99806edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.044+0000 7fe9bb200640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9bc0ce2b0 0x7fe9bc0ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.044+0000 7fe9bb200640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9bc0cd180 0x7fe9bc0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.044+0000 7fe9bb200640 1 --2- 172.21.15.131:0/3349684778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9bc072620 0x7fe9bc13b3b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.044+0000 7fe9bb200640 1 -- 172.21.15.131:0/3349684778 >> 172.21.15.131:0/3349684778 conn(0x7fe9bc068090 msgr2=0x7fe9bc0b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.044+0000 7fe9bb200640 1 -- 172.21.15.131:0/3349684778 shutdown_connections 2024-04-28T12:39:15.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.044+0000 7fe9bb200640 1 -- 172.21.15.131:0/3349684778 wait complete. 2024-04-28T12:39:15.060 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-28T12:39:15.060 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:39:15.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.267+0000 7f7e4de00640 1 Processor -- start 2024-04-28T12:39:15.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.267+0000 7f7e4de00640 1 -- start start 2024-04-28T12:39:15.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.269+0000 7f7e4de00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7e480af710 0x7f7e480afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:15.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.269+0000 7f7e4de00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7e480cf600 0x7f7e480cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:15.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.269+0000 7f7e4de00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e48072db0 con 0x7f7e4815bb90 2024-04-28T12:39:15.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.269+0000 7f7e4de00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7e480cec10 con 0x7f7e480af710 2024-04-28T12:39:15.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.269+0000 7f7e4de00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f7e480b00b0 con 0x7f7e480cf600 2024-04-28T12:39:15.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.269+0000 7f7e47e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7e480af710 0x7f7e480afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:15.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.269+0000 7f7e47e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7e480af710 0x7f7e480afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57342/0 (socket says 172.21.15.131:57342) 2024-04-28T12:39:15.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.269+0000 7f7e47e00640 1 -- 172.21.15.131:0/1041766753 learned_addr learned my addr 172.21.15.131:0/1041766753 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:15.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.269+0000 7f7e47e00640 1 -- 172.21.15.131:0/1041766753 >> v2:172.21.15.157:3300/0 conn(0x7f7e480cf600 msgr2=0x7f7e480cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:15.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.269+0000 7f7e4d400640 1 --2- 172.21.15.131:0/1041766753 >> v2:172.21.15.157:3300/0 conn(0x7f7e480cf600 0x7f7e480cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:15.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.269+0000 7f7e47e00640 1 --2- 172.21.15.131:0/1041766753 >> v2:172.21.15.157:3300/0 conn(0x7f7e480cf600 0x7f7e480cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.269+0000 7f7e47e00640 1 -- 172.21.15.131:0/1041766753 >> v1:172.21.15.157:6789/0 conn(0x7f7e4815bb90 legacy=0x7f7e480737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:15.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.270+0000 7f7e47e00640 1 -- 172.21.15.131:0/1041766753 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e480cee40 con 0x7f7e480af710 2024-04-28T12:39:15.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.270+0000 7f7e4d400640 1 --2- 172.21.15.131:0/1041766753 >> v2:172.21.15.157:3300/0 conn(0x7f7e480cf600 0x7f7e480cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:15.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.271+0000 7f7e47e00640 1 --2- 172.21.15.131:0/1041766753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e480af710 0x7f7e480afb70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7e48071b20 tx=0x7f7e3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27e5221bc8de81c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:15.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.271+0000 7f7e47400640 1 -- 172.21.15.131:0/1041766753 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e38037070 con 0x7f7e480af710 2024-04-28T12:39:15.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.271+0000 7f7e47400640 1 -- 172.21.15.131:0/1041766753 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e38032440 con 0x7f7e480af710 2024-04-28T12:39:15.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.271+0000 7f7e47400640 1 -- 172.21.15.131:0/1041766753 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e38032810 con 0x7f7e480af710 2024-04-28T12:39:15.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.271+0000 7f7e4de00640 1 -- 172.21.15.131:0/1041766753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e480af710 msgr2=0x7f7e480afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:15.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.271+0000 7f7e4de00640 1 --2- 172.21.15.131:0/1041766753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e480af710 0x7f7e480afb70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7e48071b20 tx=0x7f7e3802a230 comp rx=0 tx=0).stop 2024-04-28T12:39:15.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.272+0000 7f7e4de00640 1 -- 172.21.15.131:0/1041766753 shutdown_connections 2024-04-28T12:39:15.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.272+0000 7f7e4de00640 1 --2- 172.21.15.131:0/1041766753 >> v2:172.21.15.157:3300/0 conn(0x7f7e480cf600 0x7f7e480cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.272+0000 7f7e4de00640 1 --2- 172.21.15.131:0/1041766753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e480af710 0x7f7e480afb70 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.272+0000 7f7e4de00640 1 -- 172.21.15.131:0/1041766753 >> 172.21.15.131:0/1041766753 conn(0x7f7e48068090 msgr2=0x7f7e48068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:15.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.272+0000 7f7e4de00640 1 -- 172.21.15.131:0/1041766753 shutdown_connections 2024-04-28T12:39:15.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.272+0000 7f7e4de00640 1 -- 172.21.15.131:0/1041766753 wait complete. 2024-04-28T12:39:15.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.273+0000 7f7e4de00640 1 Processor -- start 2024-04-28T12:39:15.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.274+0000 7f7e4de00640 1 -- start start 2024-04-28T12:39:15.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.274+0000 7f7e4de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e480af710 0x7f7e4814adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:15.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.274+0000 7f7e4de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e480cf600 0x7f7e48149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:15.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.274+0000 7f7e4de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e4815bb90 0x7f7e4814f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.274+0000 7f7e4de00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7e48073eb0 con 0x7f7e4815bb90 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.274+0000 7f7e4de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7e480725a0 con 0x7f7e480cf600 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.274+0000 7f7e4de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7e48072ad0 con 0x7f7e480af710 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.275+0000 7f7e4d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e4815bb90 0x7f7e4814f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.275+0000 7f7e4d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e4815bb90 0x7f7e4814f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57352/0 (socket says 172.21.15.131:57352) 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.275+0000 7f7e4d400640 1 -- 172.21.15.131:0/3310191326 learned_addr learned my addr 172.21.15.131:0/3310191326 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.275+0000 7f7e47e00640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e480cf600 0x7f7e48149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.275+0000 7f7e4d400640 1 -- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e480af710 msgr2=0x7f7e4814adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.275+0000 7f7e4d400640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e480af710 0x7f7e4814adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.275+0000 7f7e4d400640 1 -- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e480cf600 msgr2=0x7f7e48149520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.275+0000 7f7e4d400640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e480cf600 0x7f7e48149520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.275+0000 7f7e4d400640 1 -- 172.21.15.131:0/3310191326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e481561f0 con 0x7f7e4815bb90 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.275+0000 7f7e4d400640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e4815bb90 0x7f7e4814f390 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000cef0 tx=0x7f7e4000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:15.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.276+0000 7f7e44c00640 1 -- 172.21.15.131:0/3310191326 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4001a070 con 0x7f7e4815bb90 2024-04-28T12:39:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.276+0000 7f7e44c00640 1 -- 172.21.15.131:0/3310191326 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4000d860 con 0x7f7e4815bb90 2024-04-28T12:39:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.276+0000 7f7e44c00640 1 -- 172.21.15.131:0/3310191326 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000dc50 con 0x7f7e4815bb90 2024-04-28T12:39:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.276+0000 7f7e4de00640 1 -- 172.21.15.131:0/3310191326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e4815acf0 con 0x7f7e4815bb90 2024-04-28T12:39:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.276+0000 7f7e4de00640 1 -- 172.21.15.131:0/3310191326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e4815b2b0 con 0x7f7e4815bb90 2024-04-28T12:39:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.277+0000 7f7e4de00640 1 -- 172.21.15.131:0/3310191326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e48069350 con 0x7f7e4815bb90 2024-04-28T12:39:15.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.278+0000 7f7e44c00640 1 -- 172.21.15.131:0/3310191326 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e4001d070 con 0x7f7e4815bb90 2024-04-28T12:39:15.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.279+0000 7f7e44c00640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7e2406c8f0 0x7f7e2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:15.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.279+0000 7f7e44c00640 1 -- 172.21.15.131:0/3310191326 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f7e4008c5b0 con 0x7f7e4815bb90 2024-04-28T12:39:15.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.280+0000 7f7e4ca00640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7e2406c8f0 0x7f7e2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:15.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.281+0000 7f7e4ca00640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7e2406c8f0 0x7f7e2406edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7e30006fd0 tx=0x7f7e30006bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:15.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.284+0000 7f7e44c00640 1 -- 172.21.15.131:0/3310191326 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e4005a910 con 0x7f7e4815bb90 2024-04-28T12:39:15.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.465+0000 7f7e4de00640 1 -- 172.21.15.131:0/3310191326 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e4815f380 con 0x7f7e2406c8f0 2024-04-28T12:39:15.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.486+0000 7f7e44c00640 1 -- 172.21.15.131:0/3310191326 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240074 (secure 0 0 0) 0x7f7e4815f380 con 0x7f7e2406c8f0 2024-04-28T12:39:15.487 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:15.488 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:39:15.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.492+0000 7f7e4de00640 1 -- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7e2406c8f0 msgr2=0x7f7e2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:15.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.492+0000 7f7e4de00640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7e2406c8f0 0x7f7e2406edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7e30006fd0 tx=0x7f7e30006bc0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.492+0000 7f7e4de00640 1 -- 172.21.15.131:0/3310191326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e4815bb90 msgr2=0x7f7e4814f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:15.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.492+0000 7f7e4de00640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e4815bb90 0x7f7e4814f390 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000cef0 tx=0x7f7e4000a710 comp rx=0 tx=0).stop 2024-04-28T12:39:15.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.492+0000 7f7e4de00640 1 -- 172.21.15.131:0/3310191326 shutdown_connections 2024-04-28T12:39:15.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.492+0000 7f7e4de00640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7e2406c8f0 0x7f7e2406edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.492+0000 7f7e4de00640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e4815bb90 0x7f7e4814f390 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.492+0000 7f7e4de00640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e480cf600 0x7f7e48149520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.492+0000 7f7e4de00640 1 --2- 172.21.15.131:0/3310191326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e480af710 0x7f7e4814adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.492+0000 7f7e4de00640 1 -- 172.21.15.131:0/3310191326 >> 172.21.15.131:0/3310191326 conn(0x7f7e48068090 msgr2=0x7f7e4806ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:15.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.492+0000 7f7e4de00640 1 -- 172.21.15.131:0/3310191326 shutdown_connections 2024-04-28T12:39:15.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.492+0000 7f7e4de00640 1 -- 172.21.15.131:0/3310191326 wait complete. 2024-04-28T12:39:15.508 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-28T12:39:14.958920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237096,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737624,"statfs":{"total":773094113280,"available":772851326976,"internally_reserved":0,"allocated":4399104,"data_stored":2536816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.913199"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051906+0000","last_change":"2024-04-28T12:39:09.051906+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051906+0000","last_undegraded":"2024-04-28T12:39:09.051906+0000","last_fullsized":"2024-04-28T12:39:09.051906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.055843+0000","last_change":"2024-04-28T12:39:09.055843+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.055843+0000","last_undegraded":"2024-04-28T12:39:09.055843+0000","last_fullsized":"2024-04-28T12:39:09.055843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.050841+0000","last_change":"2024-04-28T12:39:09.050841+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.050841+0000","last_undegraded":"2024-04-28T12:39:09.050841+0000","last_fullsized":"2024-04-28T12:39:09.050841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051313+0000","last_change":"2024-04-28T12:39:09.051313+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051313+0000","last_undegraded":"2024-04-28T12:39:09.051313+0000","last_fullsized":"2024-04-28T12:39:09.051313+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057502+0000","last_change":"2024-04-28T12:39:09.057502+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057502+0000","last_undegraded":"2024-04-28T12:39:09.057502+0000","last_fullsized":"2024-04-28T12:39:09.057502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.063276+0000","last_change":"2024-04-28T12:39:09.063276+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.063276+0000","last_undegraded":"2024-04-28T12:39:09.063276+0000","last_fullsized":"2024-04-28T12:39:09.063276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.050731+0000","last_change":"2024-04-28T12:39:09.050731+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.050731+0000","last_undegraded":"2024-04-28T12:39:09.050731+0000","last_fullsized":"2024-04-28T12:39:09.050731+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.052504+0000","last_change":"2024-04-28T12:39:09.052504+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.052504+0000","last_undegraded":"2024-04-28T12:39:09.052504+0000","last_fullsized":"2024-04-28T12:39:09.052504+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.058754+0000","last_change":"2024-04-28T12:39:09.058754+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.058754+0000","last_undegraded":"2024-04-28T12:39:09.058754+0000","last_fullsized":"2024-04-28T12:39:09.058754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.059079+0000","last_change":"2024-04-28T12:39:09.059079+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.059079+0000","last_undegraded":"2024-04-28T12:39:09.059079+0000","last_fullsized":"2024-04-28T12:39:09.059079+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057506+0000","last_change":"2024-04-28T12:39:09.057506+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057506+0000","last_undegraded":"2024-04-28T12:39:09.057506+0000","last_fullsized":"2024-04-28T12:39:09.057506+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057900+0000","last_change":"2024-04-28T12:39:09.057900+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057900+0000","last_undegraded":"2024-04-28T12:39:09.057900+0000","last_fullsized":"2024-04-28T12:39:09.057900+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057172+0000","last_change":"2024-04-28T12:39:09.057172+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057172+0000","last_undegraded":"2024-04-28T12:39:09.057172+0000","last_fullsized":"2024-04-28T12:39:09.057172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.050986+0000","last_change":"2024-04-28T12:39:09.050986+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.050986+0000","last_undegraded":"2024-04-28T12:39:09.050986+0000","last_fullsized":"2024-04-28T12:39:09.050986+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051153+0000","last_change":"2024-04-28T12:39:09.051153+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051153+0000","last_undegraded":"2024-04-28T12:39:09.051153+0000","last_fullsized":"2024-04-28T12:39:09.051153+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.062401+0000","last_change":"2024-04-28T12:39:09.062401+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.062401+0000","last_undegraded":"2024-04-28T12:39:09.062401+0000","last_fullsized":"2024-04-28T12:39:09.062401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.059056+0000","last_change":"2024-04-28T12:39:09.059056+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.059056+0000","last_undegraded":"2024-04-28T12:39:09.059056+0000","last_fullsized":"2024-04-28T12:39:09.059056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.052246+0000","last_change":"2024-04-28T12:39:09.052246+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.052246+0000","last_undegraded":"2024-04-28T12:39:09.052246+0000","last_fullsized":"2024-04-28T12:39:09.052246+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069983+0000","last_change":"2024-04-28T12:39:09.069983+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069983+0000","last_undegraded":"2024-04-28T12:39:09.069983+0000","last_fullsized":"2024-04-28T12:39:09.069983+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.050616+0000","last_change":"2024-04-28T12:39:09.050616+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.050616+0000","last_undegraded":"2024-04-28T12:39:09.050616+0000","last_fullsized":"2024-04-28T12:39:09.050616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069600+0000","last_change":"2024-04-28T12:39:09.069600+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069600+0000","last_undegraded":"2024-04-28T12:39:09.069600+0000","last_fullsized":"2024-04-28T12:39:09.069600+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069293+0000","last_change":"2024-04-28T12:39:09.069293+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069293+0000","last_undegraded":"2024-04-28T12:39:09.069293+0000","last_fullsized":"2024-04-28T12:39:09.069293+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.062643+0000","last_change":"2024-04-28T12:39:09.062643+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.062643+0000","last_undegraded":"2024-04-28T12:39:09.062643+0000","last_fullsized":"2024-04-28T12:39:09.062643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069283+0000","last_change":"2024-04-28T12:39:09.069283+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069283+0000","last_undegraded":"2024-04-28T12:39:09.069283+0000","last_fullsized":"2024-04-28T12:39:09.069283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057028+0000","last_change":"2024-04-28T12:39:09.057028+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057028+0000","last_undegraded":"2024-04-28T12:39:09.057028+0000","last_fullsized":"2024-04-28T12:39:09.057028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069760+0000","last_change":"2024-04-28T12:39:09.069760+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069760+0000","last_undegraded":"2024-04-28T12:39:09.069760+0000","last_fullsized":"2024-04-28T12:39:09.069760+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.055913+0000","last_change":"2024-04-28T12:39:09.055913+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.055913+0000","last_undegraded":"2024-04-28T12:39:09.055913+0000","last_fullsized":"2024-04-28T12:39:09.055913+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.055649+0000","last_change":"2024-04-28T12:39:09.055649+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.055649+0000","last_undegraded":"2024-04-28T12:39:09.055649+0000","last_fullsized":"2024-04-28T12:39:09.055649+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051406+0000","last_change":"2024-04-28T12:39:09.051406+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051406+0000","last_undegraded":"2024-04-28T12:39:09.051406+0000","last_fullsized":"2024-04-28T12:39:09.051406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051769+0000","last_change":"2024-04-28T12:39:09.051769+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051769+0000","last_undegraded":"2024-04-28T12:39:09.051769+0000","last_fullsized":"2024-04-28T12:39:09.051769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057585+0000","last_change":"2024-04-28T12:39:09.057585+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057585+0000","last_undegraded":"2024-04-28T12:39:09.057585+0000","last_fullsized":"2024-04-28T12:39:09.057585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.063490+0000","last_change":"2024-04-28T12:39:09.063490+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.063490+0000","last_undegraded":"2024-04-28T12:39:09.063490+0000","last_fullsized":"2024-04-28T12:39:09.063490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-28T12:39:09.064614+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:39:09.064614+0000","last_peered":"2024-04-28T12:39:09.064614+0000","last_clean":"2024-04-28T12:39:09.064614+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:39:09.064614+0000","last_undegraded":"2024-04-28T12:39:09.064614+0000","last_fullsized":"2024-04-28T12:39:09.064614+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:39:15.513 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:39:15.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.757+0000 7f3533200640 1 Processor -- start 2024-04-28T12:39:15.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.757+0000 7f3533200640 1 -- start start 2024-04-28T12:39:15.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.759+0000 7f3533200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f352c072b70 0x7f352c0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:15.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.759+0000 7f3533200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f352c0d1060 0x7f352c06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:15.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.759+0000 7f3533200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f352c146ba0 con 0x7f352c1467c0 2024-04-28T12:39:15.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.759+0000 7f3533200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f352c073dd0 con 0x7f352c0d1060 2024-04-28T12:39:15.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.759+0000 7f3533200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f352c0731c0 con 0x7f352c072b70 2024-04-28T12:39:15.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.759+0000 7f3531200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f352c0d1060 0x7f352c06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:15.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.759+0000 7f3531200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f352c0d1060 0x7f352c06dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57360/0 (socket says 172.21.15.131:57360) 2024-04-28T12:39:15.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.759+0000 7f3531200640 1 -- 172.21.15.131:0/2684744898 learned_addr learned my addr 172.21.15.131:0/2684744898 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:15.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.759+0000 7f352b400640 1 --2- 172.21.15.131:0/2684744898 >> v2:172.21.15.157:3300/0 conn(0x7f352c072b70 0x7f352c0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:15.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.760+0000 7f3531200640 1 -- 172.21.15.131:0/2684744898 >> v2:172.21.15.157:3300/0 conn(0x7f352c072b70 msgr2=0x7f352c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:15.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.760+0000 7f3531200640 1 --2- 172.21.15.131:0/2684744898 >> v2:172.21.15.157:3300/0 conn(0x7f352c072b70 0x7f352c0d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.760+0000 7f3531200640 1 -- 172.21.15.131:0/2684744898 >> v1:172.21.15.157:6789/0 conn(0x7f352c1467c0 legacy=0x7f352c0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:15.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.760+0000 7f3531200640 1 -- 172.21.15.131:0/2684744898 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c0734a0 con 0x7f352c0d1060 2024-04-28T12:39:15.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.761+0000 7f3531200640 1 --2- 172.21.15.131:0/2684744898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f352c0d1060 0x7f352c06dff0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f352c072320 tx=0x7f352402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cce7af4fcd3721 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:15.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.761+0000 7f352aa00640 1 -- 172.21.15.131:0/2684744898 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524036070 con 0x7f352c0d1060 2024-04-28T12:39:15.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.761+0000 7f352aa00640 1 -- 172.21.15.131:0/2684744898 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3524031600 con 0x7f352c0d1060 2024-04-28T12:39:15.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.761+0000 7f352aa00640 1 -- 172.21.15.131:0/2684744898 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35240319d0 con 0x7f352c0d1060 2024-04-28T12:39:15.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.762+0000 7f3533200640 1 -- 172.21.15.131:0/2684744898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f352c0d1060 msgr2=0x7f352c06dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:15.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.762+0000 7f3533200640 1 --2- 172.21.15.131:0/2684744898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f352c0d1060 0x7f352c06dff0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f352c072320 tx=0x7f352402a6f0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.762+0000 7f3533200640 1 -- 172.21.15.131:0/2684744898 shutdown_connections 2024-04-28T12:39:15.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.762+0000 7f3533200640 1 --2- 172.21.15.131:0/2684744898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f352c0d1060 0x7f352c06dff0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.762+0000 7f3533200640 1 --2- 172.21.15.131:0/2684744898 >> v2:172.21.15.157:3300/0 conn(0x7f352c072b70 0x7f352c0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.762+0000 7f3533200640 1 -- 172.21.15.131:0/2684744898 >> 172.21.15.131:0/2684744898 conn(0x7f352c0b2900 msgr2=0x7f352c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:15.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.762+0000 7f3533200640 1 -- 172.21.15.131:0/2684744898 shutdown_connections 2024-04-28T12:39:15.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.762+0000 7f3533200640 1 -- 172.21.15.131:0/2684744898 wait complete. 2024-04-28T12:39:15.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.764+0000 7f3533200640 1 Processor -- start 2024-04-28T12:39:15.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.764+0000 7f3533200640 1 -- start start 2024-04-28T12:39:15.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.765+0000 7f3533200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f352c072b70 0x7f352c13f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:15.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.765+0000 7f3533200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f352c0d1060 0x7f352c13b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:15.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.765+0000 7f3533200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f352c1467c0 0x7f352c0d0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:15.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.765+0000 7f3533200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f352c06e530 con 0x7f352c072b70 2024-04-28T12:39:15.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.765+0000 7f3533200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f352c073a60 con 0x7f352c0d1060 2024-04-28T12:39:15.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.765+0000 7f3533200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f352c06e6b0 con 0x7f352c1467c0 2024-04-28T12:39:15.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.765+0000 7f352be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f352c072b70 0x7f352c13f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:15.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.766+0000 7f352be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f352c072b70 0x7f352c13f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57366/0 (socket says 172.21.15.131:57366) 2024-04-28T12:39:15.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.766+0000 7f352be00640 1 -- 172.21.15.131:0/2439978799 learned_addr learned my addr 172.21.15.131:0/2439978799 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:15.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.766+0000 7f352be00640 1 -- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f352c1467c0 msgr2=0x7f352c0d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:15.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.766+0000 7f3531200640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f352c1467c0 0x7f352c0d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:15.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.766+0000 7f352be00640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f352c1467c0 0x7f352c0d0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.766+0000 7f352b400640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f352c0d1060 0x7f352c13b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:15.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.766+0000 7f352be00640 1 -- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f352c0d1060 msgr2=0x7f352c13b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:15.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.766+0000 7f352be00640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f352c0d1060 0x7f352c13b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.766+0000 7f352be00640 1 -- 172.21.15.131:0/2439978799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c0cefb0 con 0x7f352c072b70 2024-04-28T12:39:15.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.766+0000 7f352be00640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f352c072b70 0x7f352c13f290 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f351c00b3f0 tx=0x7f351c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:15.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.767+0000 7f3523e00640 1 -- 172.21.15.131:0/2439978799 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c015020 con 0x7f352c072b70 2024-04-28T12:39:15.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.767+0000 7f3523e00640 1 -- 172.21.15.131:0/2439978799 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f351c011830 con 0x7f352c072b70 2024-04-28T12:39:15.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.767+0000 7f3523e00640 1 -- 172.21.15.131:0/2439978799 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c011c20 con 0x7f352c072b70 2024-04-28T12:39:15.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.767+0000 7f3533200640 1 -- 172.21.15.131:0/2439978799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352c0cf290 con 0x7f352c072b70 2024-04-28T12:39:15.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.767+0000 7f3533200640 1 -- 172.21.15.131:0/2439978799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352c0cf850 con 0x7f352c072b70 2024-04-28T12:39:15.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.768+0000 7f3533200640 1 -- 172.21.15.131:0/2439978799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352c0cfc90 con 0x7f352c072b70 2024-04-28T12:39:15.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.769+0000 7f3523e00640 1 -- 172.21.15.131:0/2439978799 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f351c018020 con 0x7f352c072b70 2024-04-28T12:39:15.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.770+0000 7f3523e00640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f350c06c8f0 0x7f350c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:15.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.770+0000 7f3523e00640 1 -- 172.21.15.131:0/2439978799 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f351c08b980 con 0x7f352c072b70 2024-04-28T12:39:15.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.771+0000 7f352b400640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f350c06c8f0 0x7f350c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:15.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.772+0000 7f352b400640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f350c06c8f0 0x7f350c06edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f352c143440 tx=0x7f3518008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:15.769 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:39:15.774 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:39:15.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.775+0000 7f3523e00640 1 -- 172.21.15.131:0/2439978799 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f351c01f030 con 0x7f352c072b70 2024-04-28T12:39:15.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.978+0000 7f3533200640 1 -- 172.21.15.131:0/2439978799 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f352c13f7d0 con 0x7f350c06c8f0 2024-04-28T12:39:15.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:15.996+0000 7f3523e00640 1 -- 172.21.15.131:0/2439978799 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240074 (secure 0 0 0) 0x7f3518009d60 con 0x7f350c06c8f0 2024-04-28T12:39:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:15.997 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:39:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.002+0000 7f3533200640 1 -- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f350c06c8f0 msgr2=0x7f350c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.002+0000 7f3533200640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f350c06c8f0 0x7f350c06edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f352c143440 tx=0x7f3518008040 comp rx=0 tx=0).stop 2024-04-28T12:39:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.002+0000 7f3533200640 1 -- 172.21.15.131:0/2439978799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f352c072b70 msgr2=0x7f352c13f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.002+0000 7f3533200640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f352c072b70 0x7f352c13f290 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f351c00b3f0 tx=0x7f351c00b8b0 comp rx=0 tx=0).stop 2024-04-28T12:39:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.002+0000 7f3533200640 1 -- 172.21.15.131:0/2439978799 shutdown_connections 2024-04-28T12:39:15.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.002+0000 7f3533200640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f350c06c8f0 0x7f350c06edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.002+0000 7f3533200640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f352c1467c0 0x7f352c0d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.002+0000 7f3533200640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f352c0d1060 0x7f352c13b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.002+0000 7f3533200640 1 --2- 172.21.15.131:0/2439978799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f352c072b70 0x7f352c13f290 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.002+0000 7f3533200640 1 -- 172.21.15.131:0/2439978799 >> 172.21.15.131:0/2439978799 conn(0x7f352c0b2900 msgr2=0x7f352c15f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:16.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.002+0000 7f3533200640 1 -- 172.21.15.131:0/2439978799 shutdown_connections 2024-04-28T12:39:16.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.002+0000 7f3533200640 1 -- 172.21.15.131:0/2439978799 wait complete. 2024-04-28T12:39:16.018 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-28T12:39:14.958920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237096,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737624,"statfs":{"total":773094113280,"available":772851326976,"internally_reserved":0,"allocated":4399104,"data_stored":2536816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.913199"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051906+0000","last_change":"2024-04-28T12:39:09.051906+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051906+0000","last_undegraded":"2024-04-28T12:39:09.051906+0000","last_fullsized":"2024-04-28T12:39:09.051906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.055843+0000","last_change":"2024-04-28T12:39:09.055843+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.055843+0000","last_undegraded":"2024-04-28T12:39:09.055843+0000","last_fullsized":"2024-04-28T12:39:09.055843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.050841+0000","last_change":"2024-04-28T12:39:09.050841+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.050841+0000","last_undegraded":"2024-04-28T12:39:09.050841+0000","last_fullsized":"2024-04-28T12:39:09.050841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051313+0000","last_change":"2024-04-28T12:39:09.051313+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051313+0000","last_undegraded":"2024-04-28T12:39:09.051313+0000","last_fullsized":"2024-04-28T12:39:09.051313+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057502+0000","last_change":"2024-04-28T12:39:09.057502+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057502+0000","last_undegraded":"2024-04-28T12:39:09.057502+0000","last_fullsized":"2024-04-28T12:39:09.057502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.063276+0000","last_change":"2024-04-28T12:39:09.063276+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.063276+0000","last_undegraded":"2024-04-28T12:39:09.063276+0000","last_fullsized":"2024-04-28T12:39:09.063276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.050731+0000","last_change":"2024-04-28T12:39:09.050731+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.050731+0000","last_undegraded":"2024-04-28T12:39:09.050731+0000","last_fullsized":"2024-04-28T12:39:09.050731+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.052504+0000","last_change":"2024-04-28T12:39:09.052504+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.052504+0000","last_undegraded":"2024-04-28T12:39:09.052504+0000","last_fullsized":"2024-04-28T12:39:09.052504+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.058754+0000","last_change":"2024-04-28T12:39:09.058754+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.058754+0000","last_undegraded":"2024-04-28T12:39:09.058754+0000","last_fullsized":"2024-04-28T12:39:09.058754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.059079+0000","last_change":"2024-04-28T12:39:09.059079+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.059079+0000","last_undegraded":"2024-04-28T12:39:09.059079+0000","last_fullsized":"2024-04-28T12:39:09.059079+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057506+0000","last_change":"2024-04-28T12:39:09.057506+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057506+0000","last_undegraded":"2024-04-28T12:39:09.057506+0000","last_fullsized":"2024-04-28T12:39:09.057506+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057900+0000","last_change":"2024-04-28T12:39:09.057900+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057900+0000","last_undegraded":"2024-04-28T12:39:09.057900+0000","last_fullsized":"2024-04-28T12:39:09.057900+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057172+0000","last_change":"2024-04-28T12:39:09.057172+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057172+0000","last_undegraded":"2024-04-28T12:39:09.057172+0000","last_fullsized":"2024-04-28T12:39:09.057172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.050986+0000","last_change":"2024-04-28T12:39:09.050986+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.050986+0000","last_undegraded":"2024-04-28T12:39:09.050986+0000","last_fullsized":"2024-04-28T12:39:09.050986+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051153+0000","last_change":"2024-04-28T12:39:09.051153+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051153+0000","last_undegraded":"2024-04-28T12:39:09.051153+0000","last_fullsized":"2024-04-28T12:39:09.051153+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.062401+0000","last_change":"2024-04-28T12:39:09.062401+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.062401+0000","last_undegraded":"2024-04-28T12:39:09.062401+0000","last_fullsized":"2024-04-28T12:39:09.062401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.059056+0000","last_change":"2024-04-28T12:39:09.059056+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.059056+0000","last_undegraded":"2024-04-28T12:39:09.059056+0000","last_fullsized":"2024-04-28T12:39:09.059056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.052246+0000","last_change":"2024-04-28T12:39:09.052246+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.052246+0000","last_undegraded":"2024-04-28T12:39:09.052246+0000","last_fullsized":"2024-04-28T12:39:09.052246+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069983+0000","last_change":"2024-04-28T12:39:09.069983+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069983+0000","last_undegraded":"2024-04-28T12:39:09.069983+0000","last_fullsized":"2024-04-28T12:39:09.069983+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.050616+0000","last_change":"2024-04-28T12:39:09.050616+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.050616+0000","last_undegraded":"2024-04-28T12:39:09.050616+0000","last_fullsized":"2024-04-28T12:39:09.050616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069600+0000","last_change":"2024-04-28T12:39:09.069600+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069600+0000","last_undegraded":"2024-04-28T12:39:09.069600+0000","last_fullsized":"2024-04-28T12:39:09.069600+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069293+0000","last_change":"2024-04-28T12:39:09.069293+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069293+0000","last_undegraded":"2024-04-28T12:39:09.069293+0000","last_fullsized":"2024-04-28T12:39:09.069293+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.062643+0000","last_change":"2024-04-28T12:39:09.062643+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.062643+0000","last_undegraded":"2024-04-28T12:39:09.062643+0000","last_fullsized":"2024-04-28T12:39:09.062643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069283+0000","last_change":"2024-04-28T12:39:09.069283+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069283+0000","last_undegraded":"2024-04-28T12:39:09.069283+0000","last_fullsized":"2024-04-28T12:39:09.069283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057028+0000","last_change":"2024-04-28T12:39:09.057028+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057028+0000","last_undegraded":"2024-04-28T12:39:09.057028+0000","last_fullsized":"2024-04-28T12:39:09.057028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069760+0000","last_change":"2024-04-28T12:39:09.069760+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069760+0000","last_undegraded":"2024-04-28T12:39:09.069760+0000","last_fullsized":"2024-04-28T12:39:09.069760+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.055913+0000","last_change":"2024-04-28T12:39:09.055913+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.055913+0000","last_undegraded":"2024-04-28T12:39:09.055913+0000","last_fullsized":"2024-04-28T12:39:09.055913+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.055649+0000","last_change":"2024-04-28T12:39:09.055649+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.055649+0000","last_undegraded":"2024-04-28T12:39:09.055649+0000","last_fullsized":"2024-04-28T12:39:09.055649+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051406+0000","last_change":"2024-04-28T12:39:09.051406+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051406+0000","last_undegraded":"2024-04-28T12:39:09.051406+0000","last_fullsized":"2024-04-28T12:39:09.051406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051769+0000","last_change":"2024-04-28T12:39:09.051769+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051769+0000","last_undegraded":"2024-04-28T12:39:09.051769+0000","last_fullsized":"2024-04-28T12:39:09.051769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057585+0000","last_change":"2024-04-28T12:39:09.057585+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057585+0000","last_undegraded":"2024-04-28T12:39:09.057585+0000","last_fullsized":"2024-04-28T12:39:09.057585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.063490+0000","last_change":"2024-04-28T12:39:09.063490+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.063490+0000","last_undegraded":"2024-04-28T12:39:09.063490+0000","last_fullsized":"2024-04-28T12:39:09.063490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-28T12:39:09.064614+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:39:09.064614+0000","last_peered":"2024-04-28T12:39:09.064614+0000","last_clean":"2024-04-28T12:39:09.064614+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:39:09.064614+0000","last_undegraded":"2024-04-28T12:39:09.064614+0000","last_fullsized":"2024-04-28T12:39:09.064614+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:39:16.022 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:39:16.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.272+0000 7ff780c00640 1 Processor -- start 2024-04-28T12:39:16.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.272+0000 7ff780c00640 1 -- start start 2024-04-28T12:39:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.274+0000 7ff780c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff77c0d1620 0x7ff77c0d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.274+0000 7ff780c00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ff77c0d1a00 0x7ff77c0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.274+0000 7ff780c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff77c0d0480 con 0x7ff77c0b50b0 2024-04-28T12:39:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.274+0000 7ff780c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff77c0d0ce0 con 0x7ff77c0d1620 2024-04-28T12:39:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.274+0000 7ff780c00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ff77c0d0720 con 0x7ff77c0d1a00 2024-04-28T12:39:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.274+0000 7ff77b400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff77c0d1620 0x7ff77c0d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.274+0000 7ff77b400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff77c0d1620 0x7ff77c0d5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57378/0 (socket says 172.21.15.131:57378) 2024-04-28T12:39:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.274+0000 7ff77b400640 1 -- 172.21.15.131:0/1238783244 learned_addr learned my addr 172.21.15.131:0/1238783244 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.275+0000 7ff77b400640 1 -- 172.21.15.131:0/1238783244 >> v2:172.21.15.157:3300/0 conn(0x7ff77c0d1a00 msgr2=0x7ff77c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.275+0000 7ff77b400640 1 --2- 172.21.15.131:0/1238783244 >> v2:172.21.15.157:3300/0 conn(0x7ff77c0d1a00 0x7ff77c0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.275+0000 7ff77b400640 1 -- 172.21.15.131:0/1238783244 >> v1:172.21.15.157:6789/0 conn(0x7ff77c0b50b0 legacy=0x7ff77c0b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.275+0000 7ff77b400640 1 -- 172.21.15.131:0/1238783244 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c0d0a00 con 0x7ff77c0d1620 2024-04-28T12:39:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.276+0000 7ff77b400640 1 --2- 172.21.15.131:0/1238783244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff77c0d1620 0x7ff77c0d5750 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff77c08d330 tx=0x7ff76c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f730d0216eb8324 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:16.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.276+0000 7ff77aa00640 1 -- 172.21.15.131:0/1238783244 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c037070 con 0x7ff77c0d1620 2024-04-28T12:39:16.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.276+0000 7ff77aa00640 1 -- 172.21.15.131:0/1238783244 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c032440 con 0x7ff77c0d1620 2024-04-28T12:39:16.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.276+0000 7ff77aa00640 1 -- 172.21.15.131:0/1238783244 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c032810 con 0x7ff77c0d1620 2024-04-28T12:39:16.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.277+0000 7ff780c00640 1 -- 172.21.15.131:0/1238783244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff77c0d1620 msgr2=0x7ff77c0d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:16.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.277+0000 7ff780c00640 1 --2- 172.21.15.131:0/1238783244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff77c0d1620 0x7ff77c0d5750 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff77c08d330 tx=0x7ff76c02a180 comp rx=0 tx=0).stop 2024-04-28T12:39:16.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.277+0000 7ff780c00640 1 -- 172.21.15.131:0/1238783244 shutdown_connections 2024-04-28T12:39:16.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.277+0000 7ff780c00640 1 --2- 172.21.15.131:0/1238783244 >> v2:172.21.15.157:3300/0 conn(0x7ff77c0d1a00 0x7ff77c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.277+0000 7ff780c00640 1 --2- 172.21.15.131:0/1238783244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff77c0d1620 0x7ff77c0d5750 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.277+0000 7ff780c00640 1 -- 172.21.15.131:0/1238783244 >> 172.21.15.131:0/1238783244 conn(0x7ff77c0d4c30 msgr2=0x7ff77c0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:16.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.277+0000 7ff780c00640 1 -- 172.21.15.131:0/1238783244 shutdown_connections 2024-04-28T12:39:16.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.277+0000 7ff780c00640 1 -- 172.21.15.131:0/1238783244 wait complete. 2024-04-28T12:39:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.278+0000 7ff780c00640 1 Processor -- start 2024-04-28T12:39:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.279+0000 7ff780c00640 1 -- start start 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.279+0000 7ff780c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77c0b50b0 0x7ff77c0cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff780c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff77c0d1620 0x7ff77c09e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff780c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77c0d1a00 0x7ff77c154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff780c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c0b5c50 con 0x7ff77c0d1620 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff780c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c0d1e60 con 0x7ff77c0d1a00 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff780c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff77c0b5dd0 con 0x7ff77c0b50b0 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff772a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff77c0d1620 0x7ff77c09e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff772a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff77c0d1620 0x7ff77c09e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57390/0 (socket says 172.21.15.131:57390) 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff772a00640 1 -- 172.21.15.131:0/3508889405 learned_addr learned my addr 172.21.15.131:0/3508889405 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff77be00640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77c0d1a00 0x7ff77c154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff772a00640 1 -- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77c0b50b0 msgr2=0x7ff77c0cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff772a00640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77c0b50b0 0x7ff77c0cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff772a00640 1 -- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77c0d1a00 msgr2=0x7ff77c154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:16.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff772a00640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77c0d1a00 0x7ff77c154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.280+0000 7ff772a00640 1 -- 172.21.15.131:0/3508889405 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c09ef20 con 0x7ff77c0d1620 2024-04-28T12:39:16.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.281+0000 7ff772a00640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff77c0d1620 0x7ff77c09e9e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff76400b410 tx=0x7ff76400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.281+0000 7ff773e00640 1 -- 172.21.15.131:0/3508889405 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764015020 con 0x7ff77c0d1620 2024-04-28T12:39:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.281+0000 7ff773e00640 1 -- 172.21.15.131:0/3508889405 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff764011850 con 0x7ff77c0d1620 2024-04-28T12:39:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.281+0000 7ff773e00640 1 -- 172.21.15.131:0/3508889405 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764011c40 con 0x7ff77c0d1620 2024-04-28T12:39:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.281+0000 7ff780c00640 1 -- 172.21.15.131:0/3508889405 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77c0b8a10 con 0x7ff77c0d1620 2024-04-28T12:39:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.281+0000 7ff780c00640 1 -- 172.21.15.131:0/3508889405 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77c0b8fd0 con 0x7ff77c0d1620 2024-04-28T12:39:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.282+0000 7ff780c00640 1 -- 172.21.15.131:0/3508889405 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77c069350 con 0x7ff77c0d1620 2024-04-28T12:39:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.283+0000 7ff773e00640 1 -- 172.21.15.131:0/3508889405 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff764018020 con 0x7ff77c0d1620 2024-04-28T12:39:16.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.284+0000 7ff773e00640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff75806c8f0 0x7ff75806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:16.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.284+0000 7ff773e00640 1 -- 172.21.15.131:0/3508889405 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7ff76408bc70 con 0x7ff77c0d1620 2024-04-28T12:39:16.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.285+0000 7ff77b400640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff75806c8f0 0x7ff75806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:16.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.286+0000 7ff77b400640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff75806c8f0 0x7ff75806edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff76c034000 tx=0x7ff76c02c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:16.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.289+0000 7ff773e00640 1 -- 172.21.15.131:0/3508889405 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff76401f020 con 0x7ff77c0d1620 2024-04-28T12:39:16.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.484+0000 7ff780c00640 1 -- 172.21.15.131:0/3508889405 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff77c0b1860 con 0x7ff75806c8f0 2024-04-28T12:39:16.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.502+0000 7ff773e00640 1 -- 172.21.15.131:0/3508889405 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240074 (secure 0 0 0) 0x7ff77c0b1860 con 0x7ff75806c8f0 2024-04-28T12:39:16.502 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:39:16.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.507+0000 7ff780c00640 1 -- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff75806c8f0 msgr2=0x7ff75806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:16.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.507+0000 7ff780c00640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff75806c8f0 0x7ff75806edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff76c034000 tx=0x7ff76c02c000 comp rx=0 tx=0).stop 2024-04-28T12:39:16.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.507+0000 7ff780c00640 1 -- 172.21.15.131:0/3508889405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff77c0d1620 msgr2=0x7ff77c09e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.507+0000 7ff780c00640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff77c0d1620 0x7ff77c09e9e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff76400b410 tx=0x7ff76400b8d0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.508+0000 7ff780c00640 1 -- 172.21.15.131:0/3508889405 shutdown_connections 2024-04-28T12:39:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.508+0000 7ff780c00640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ff75806c8f0 0x7ff75806edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.508+0000 7ff780c00640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff77c0d1a00 0x7ff77c154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.508+0000 7ff780c00640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff77c0d1620 0x7ff77c09e9e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.508+0000 7ff780c00640 1 --2- 172.21.15.131:0/3508889405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff77c0b50b0 0x7ff77c0cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.508+0000 7ff780c00640 1 -- 172.21.15.131:0/3508889405 >> 172.21.15.131:0/3508889405 conn(0x7ff77c0d4c30 msgr2=0x7ff77c0b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.508+0000 7ff780c00640 1 -- 172.21.15.131:0/3508889405 shutdown_connections 2024-04-28T12:39:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.508+0000 7ff780c00640 1 -- 172.21.15.131:0/3508889405 wait complete. 2024-04-28T12:39:16.522 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-28T12:39:14.958920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237096,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737624,"statfs":{"total":773094113280,"available":772851326976,"internally_reserved":0,"allocated":4399104,"data_stored":2536816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.913199"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051906+0000","last_change":"2024-04-28T12:39:09.051906+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051906+0000","last_undegraded":"2024-04-28T12:39:09.051906+0000","last_fullsized":"2024-04-28T12:39:09.051906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.055843+0000","last_change":"2024-04-28T12:39:09.055843+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.055843+0000","last_undegraded":"2024-04-28T12:39:09.055843+0000","last_fullsized":"2024-04-28T12:39:09.055843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.050841+0000","last_change":"2024-04-28T12:39:09.050841+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.050841+0000","last_undegraded":"2024-04-28T12:39:09.050841+0000","last_fullsized":"2024-04-28T12:39:09.050841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051313+0000","last_change":"2024-04-28T12:39:09.051313+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051313+0000","last_undegraded":"2024-04-28T12:39:09.051313+0000","last_fullsized":"2024-04-28T12:39:09.051313+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057502+0000","last_change":"2024-04-28T12:39:09.057502+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057502+0000","last_undegraded":"2024-04-28T12:39:09.057502+0000","last_fullsized":"2024-04-28T12:39:09.057502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.063276+0000","last_change":"2024-04-28T12:39:09.063276+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.063276+0000","last_undegraded":"2024-04-28T12:39:09.063276+0000","last_fullsized":"2024-04-28T12:39:09.063276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.050731+0000","last_change":"2024-04-28T12:39:09.050731+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.050731+0000","last_undegraded":"2024-04-28T12:39:09.050731+0000","last_fullsized":"2024-04-28T12:39:09.050731+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.052504+0000","last_change":"2024-04-28T12:39:09.052504+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.052504+0000","last_undegraded":"2024-04-28T12:39:09.052504+0000","last_fullsized":"2024-04-28T12:39:09.052504+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.058754+0000","last_change":"2024-04-28T12:39:09.058754+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.058754+0000","last_undegraded":"2024-04-28T12:39:09.058754+0000","last_fullsized":"2024-04-28T12:39:09.058754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.059079+0000","last_change":"2024-04-28T12:39:09.059079+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.059079+0000","last_undegraded":"2024-04-28T12:39:09.059079+0000","last_fullsized":"2024-04-28T12:39:09.059079+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057506+0000","last_change":"2024-04-28T12:39:09.057506+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057506+0000","last_undegraded":"2024-04-28T12:39:09.057506+0000","last_fullsized":"2024-04-28T12:39:09.057506+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057900+0000","last_change":"2024-04-28T12:39:09.057900+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057900+0000","last_undegraded":"2024-04-28T12:39:09.057900+0000","last_fullsized":"2024-04-28T12:39:09.057900+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057172+0000","last_change":"2024-04-28T12:39:09.057172+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057172+0000","last_undegraded":"2024-04-28T12:39:09.057172+0000","last_fullsized":"2024-04-28T12:39:09.057172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.050986+0000","last_change":"2024-04-28T12:39:09.050986+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.050986+0000","last_undegraded":"2024-04-28T12:39:09.050986+0000","last_fullsized":"2024-04-28T12:39:09.050986+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051153+0000","last_change":"2024-04-28T12:39:09.051153+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051153+0000","last_undegraded":"2024-04-28T12:39:09.051153+0000","last_fullsized":"2024-04-28T12:39:09.051153+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.062401+0000","last_change":"2024-04-28T12:39:09.062401+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.062401+0000","last_undegraded":"2024-04-28T12:39:09.062401+0000","last_fullsized":"2024-04-28T12:39:09.062401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.059056+0000","last_change":"2024-04-28T12:39:09.059056+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.059056+0000","last_undegraded":"2024-04-28T12:39:09.059056+0000","last_fullsized":"2024-04-28T12:39:09.059056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.052246+0000","last_change":"2024-04-28T12:39:09.052246+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.052246+0000","last_undegraded":"2024-04-28T12:39:09.052246+0000","last_fullsized":"2024-04-28T12:39:09.052246+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069983+0000","last_change":"2024-04-28T12:39:09.069983+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069983+0000","last_undegraded":"2024-04-28T12:39:09.069983+0000","last_fullsized":"2024-04-28T12:39:09.069983+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.050616+0000","last_change":"2024-04-28T12:39:09.050616+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.050616+0000","last_undegraded":"2024-04-28T12:39:09.050616+0000","last_fullsized":"2024-04-28T12:39:09.050616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069600+0000","last_change":"2024-04-28T12:39:09.069600+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069600+0000","last_undegraded":"2024-04-28T12:39:09.069600+0000","last_fullsized":"2024-04-28T12:39:09.069600+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069293+0000","last_change":"2024-04-28T12:39:09.069293+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069293+0000","last_undegraded":"2024-04-28T12:39:09.069293+0000","last_fullsized":"2024-04-28T12:39:09.069293+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.062643+0000","last_change":"2024-04-28T12:39:09.062643+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.062643+0000","last_undegraded":"2024-04-28T12:39:09.062643+0000","last_fullsized":"2024-04-28T12:39:09.062643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069283+0000","last_change":"2024-04-28T12:39:09.069283+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069283+0000","last_undegraded":"2024-04-28T12:39:09.069283+0000","last_fullsized":"2024-04-28T12:39:09.069283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057028+0000","last_change":"2024-04-28T12:39:09.057028+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057028+0000","last_undegraded":"2024-04-28T12:39:09.057028+0000","last_fullsized":"2024-04-28T12:39:09.057028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.069760+0000","last_change":"2024-04-28T12:39:09.069760+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.069760+0000","last_undegraded":"2024-04-28T12:39:09.069760+0000","last_fullsized":"2024-04-28T12:39:09.069760+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.055913+0000","last_change":"2024-04-28T12:39:09.055913+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.055913+0000","last_undegraded":"2024-04-28T12:39:09.055913+0000","last_fullsized":"2024-04-28T12:39:09.055913+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.055649+0000","last_change":"2024-04-28T12:39:09.055649+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.055649+0000","last_undegraded":"2024-04-28T12:39:09.055649+0000","last_fullsized":"2024-04-28T12:39:09.055649+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051406+0000","last_change":"2024-04-28T12:39:09.051406+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051406+0000","last_undegraded":"2024-04-28T12:39:09.051406+0000","last_fullsized":"2024-04-28T12:39:09.051406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.051769+0000","last_change":"2024-04-28T12:39:09.051769+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.051769+0000","last_undegraded":"2024-04-28T12:39:09.051769+0000","last_fullsized":"2024-04-28T12:39:09.051769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.057585+0000","last_change":"2024-04-28T12:39:09.057585+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.057585+0000","last_undegraded":"2024-04-28T12:39:09.057585+0000","last_fullsized":"2024-04-28T12:39:09.057585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-28T12:39:10.044250+0000","last_change":"2024-04-28T12:39:10.044250+0000","last_active":"2024-04-28T12:39:10.044250+0000","last_peered":"2024-04-28T12:39:10.044250+0000","last_clean":"2024-04-28T12:39:10.044250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:10.044250+0000","last_undegraded":"2024-04-28T12:39:10.044250+0000","last_fullsized":"2024-04-28T12:39:10.044250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":35,"state":"creating+peering","last_fresh":"2024-04-28T12:39:09.063490+0000","last_change":"2024-04-28T12:39:09.063490+0000","last_active":"2024-04-28T12:39:09.029756+0000","last_peered":"2024-04-28T12:39:09.029756+0000","last_clean":"2024-04-28T12:39:09.029756+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-28T12:39:09.063490+0000","last_undegraded":"2024-04-28T12:39:09.063490+0000","last_fullsized":"2024-04-28T12:39:09.063490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-28T12:39:09.064614+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:39:09.064614+0000","last_peered":"2024-04-28T12:39:09.064614+0000","last_clean":"2024-04-28T12:39:09.064614+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:39:09.064614+0000","last_undegraded":"2024-04-28T12:39:09.064614+0000","last_fullsized":"2024-04-28T12:39:09.064614+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":169510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":995328,"data_stored":759878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:39:16.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-04-28T12:39:16.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-04-28T12:39:16.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-04-28T12:39:16.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-04-28T12:39:16.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-04-28T12:39:16.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-04-28T12:39:16.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-04-28T12:39:16.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-04-28T12:39:16.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-04-28T12:39:16.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-04-28T12:39:16.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-04-28T12:39:16.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-04-28T12:39:16.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-04-28T12:39:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-04-28T12:39:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-04-28T12:39:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-04-28T12:39:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-04-28T12:39:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-04-28T12:39:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-04-28T12:39:16.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-04-28T12:39:16.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-04-28T12:39:16.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-04-28T12:39:16.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-04-28T12:39:16.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-04-28T12:39:16.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-04-28T12:39:16.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-04-28T12:39:16.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-04-28T12:39:16.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-04-28T12:39:16.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-04-28T12:39:16.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-04-28T12:39:16.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-04-28T12:39:16.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-04-28T12:39:16.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-04-28T12:39:16.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-28T12:39:16.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.051906+0000', 'last_change': '2024-04-28T12:39:09.051906+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.051906+0000', 'last_undegraded': '2024-04-28T12:39:09.051906+0000', 'last_fullsized': '2024-04-28T12:39:09.051906+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-28T12:39:16.532 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-28T12:39:16.532 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.533 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-28T12:39:16.533 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.055843+0000', 'last_change': '2024-04-28T12:39:09.055843+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.055843+0000', 'last_undegraded': '2024-04-28T12:39:09.055843+0000', 'last_fullsized': '2024-04-28T12:39:09.055843+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-28T12:39:16.533 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-28T12:39:16.533 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.533 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-28T12:39:16.533 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.050841+0000', 'last_change': '2024-04-28T12:39:09.050841+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.050841+0000', 'last_undegraded': '2024-04-28T12:39:09.050841+0000', 'last_fullsized': '2024-04-28T12:39:09.050841+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-28T12:39:16.533 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-28T12:39:16.533 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.533 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-28T12:39:16.533 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.051313+0000', 'last_change': '2024-04-28T12:39:09.051313+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.051313+0000', 'last_undegraded': '2024-04-28T12:39:09.051313+0000', 'last_fullsized': '2024-04-28T12:39:09.051313+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-28T12:39:16.533 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-28T12:39:16.534 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.534 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-28T12:39:16.534 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.057502+0000', 'last_change': '2024-04-28T12:39:09.057502+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.057502+0000', 'last_undegraded': '2024-04-28T12:39:09.057502+0000', 'last_fullsized': '2024-04-28T12:39:09.057502+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-28T12:39:16.534 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-04-28T12:39:16.534 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.534 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-28T12:39:16.534 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.063276+0000', 'last_change': '2024-04-28T12:39:09.063276+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.063276+0000', 'last_undegraded': '2024-04-28T12:39:09.063276+0000', 'last_fullsized': '2024-04-28T12:39:09.063276+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-28T12:39:16.534 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-04-28T12:39:16.534 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.534 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-28T12:39:16.534 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.050731+0000', 'last_change': '2024-04-28T12:39:09.050731+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.050731+0000', 'last_undegraded': '2024-04-28T12:39:09.050731+0000', 'last_fullsized': '2024-04-28T12:39:09.050731+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-28T12:39:16.535 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-28T12:39:16.535 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.535 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-28T12:39:16.535 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.052504+0000', 'last_change': '2024-04-28T12:39:09.052504+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.052504+0000', 'last_undegraded': '2024-04-28T12:39:09.052504+0000', 'last_fullsized': '2024-04-28T12:39:09.052504+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-28T12:39:16.535 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-28T12:39:16.535 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.535 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-28T12:39:16.535 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.058754+0000', 'last_change': '2024-04-28T12:39:09.058754+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.058754+0000', 'last_undegraded': '2024-04-28T12:39:09.058754+0000', 'last_fullsized': '2024-04-28T12:39:09.058754+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-28T12:39:16.535 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-04-28T12:39:16.535 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.535 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-28T12:39:16.535 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.059079+0000', 'last_change': '2024-04-28T12:39:09.059079+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.059079+0000', 'last_undegraded': '2024-04-28T12:39:09.059079+0000', 'last_fullsized': '2024-04-28T12:39:09.059079+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-28T12:39:16.536 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-28T12:39:16.536 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.536 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-28T12:39:16.536 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.057506+0000', 'last_change': '2024-04-28T12:39:09.057506+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.057506+0000', 'last_undegraded': '2024-04-28T12:39:09.057506+0000', 'last_fullsized': '2024-04-28T12:39:09.057506+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-28T12:39:16.536 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-04-28T12:39:16.536 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.536 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-28T12:39:16.536 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.057900+0000', 'last_change': '2024-04-28T12:39:09.057900+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.057900+0000', 'last_undegraded': '2024-04-28T12:39:09.057900+0000', 'last_fullsized': '2024-04-28T12:39:09.057900+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-28T12:39:16.536 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-28T12:39:16.536 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.536 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-28T12:39:16.537 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.057172+0000', 'last_change': '2024-04-28T12:39:09.057172+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.057172+0000', 'last_undegraded': '2024-04-28T12:39:09.057172+0000', 'last_fullsized': '2024-04-28T12:39:09.057172+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-28T12:39:16.537 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-28T12:39:16.537 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.537 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-28T12:39:16.537 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.050986+0000', 'last_change': '2024-04-28T12:39:09.050986+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.050986+0000', 'last_undegraded': '2024-04-28T12:39:09.050986+0000', 'last_fullsized': '2024-04-28T12:39:09.050986+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-28T12:39:16.537 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-04-28T12:39:16.537 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.537 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-28T12:39:16.537 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.051153+0000', 'last_change': '2024-04-28T12:39:09.051153+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.051153+0000', 'last_undegraded': '2024-04-28T12:39:09.051153+0000', 'last_fullsized': '2024-04-28T12:39:09.051153+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-28T12:39:16.537 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-28T12:39:16.537 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.538 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-28T12:39:16.538 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.062401+0000', 'last_change': '2024-04-28T12:39:09.062401+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.062401+0000', 'last_undegraded': '2024-04-28T12:39:09.062401+0000', 'last_fullsized': '2024-04-28T12:39:09.062401+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-28T12:39:16.538 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-28T12:39:16.538 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.538 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-28T12:39:16.538 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.059056+0000', 'last_change': '2024-04-28T12:39:09.059056+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.059056+0000', 'last_undegraded': '2024-04-28T12:39:09.059056+0000', 'last_fullsized': '2024-04-28T12:39:09.059056+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-28T12:39:16.538 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-04-28T12:39:16.538 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.538 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-28T12:39:16.538 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.052246+0000', 'last_change': '2024-04-28T12:39:09.052246+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.052246+0000', 'last_undegraded': '2024-04-28T12:39:09.052246+0000', 'last_fullsized': '2024-04-28T12:39:09.052246+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-28T12:39:16.539 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-04-28T12:39:16.539 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.539 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-28T12:39:16.539 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.069983+0000', 'last_change': '2024-04-28T12:39:09.069983+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.069983+0000', 'last_undegraded': '2024-04-28T12:39:09.069983+0000', 'last_fullsized': '2024-04-28T12:39:09.069983+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-28T12:39:16.539 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-28T12:39:16.539 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.539 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-28T12:39:16.539 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.050616+0000', 'last_change': '2024-04-28T12:39:09.050616+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.050616+0000', 'last_undegraded': '2024-04-28T12:39:09.050616+0000', 'last_fullsized': '2024-04-28T12:39:09.050616+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-28T12:39:16.539 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-04-28T12:39:16.539 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.539 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-28T12:39:16.539 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.069600+0000', 'last_change': '2024-04-28T12:39:09.069600+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.069600+0000', 'last_undegraded': '2024-04-28T12:39:09.069600+0000', 'last_fullsized': '2024-04-28T12:39:09.069600+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-28T12:39:16.540 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-28T12:39:16.540 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.540 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-28T12:39:16.540 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.069293+0000', 'last_change': '2024-04-28T12:39:09.069293+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.069293+0000', 'last_undegraded': '2024-04-28T12:39:09.069293+0000', 'last_fullsized': '2024-04-28T12:39:09.069293+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-28T12:39:16.540 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-04-28T12:39:16.540 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.540 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-28T12:39:16.540 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.062643+0000', 'last_change': '2024-04-28T12:39:09.062643+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.062643+0000', 'last_undegraded': '2024-04-28T12:39:09.062643+0000', 'last_fullsized': '2024-04-28T12:39:09.062643+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-28T12:39:16.540 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-28T12:39:16.540 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.540 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-28T12:39:16.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.069283+0000', 'last_change': '2024-04-28T12:39:09.069283+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.069283+0000', 'last_undegraded': '2024-04-28T12:39:09.069283+0000', 'last_fullsized': '2024-04-28T12:39:09.069283+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-28T12:39:16.541 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-28T12:39:16.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.541 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-28T12:39:16.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.057028+0000', 'last_change': '2024-04-28T12:39:09.057028+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.057028+0000', 'last_undegraded': '2024-04-28T12:39:09.057028+0000', 'last_fullsized': '2024-04-28T12:39:09.057028+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-28T12:39:16.541 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-04-28T12:39:16.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.541 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-28T12:39:16.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.069760+0000', 'last_change': '2024-04-28T12:39:09.069760+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.069760+0000', 'last_undegraded': '2024-04-28T12:39:09.069760+0000', 'last_fullsized': '2024-04-28T12:39:09.069760+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-28T12:39:16.541 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-28T12:39:16.541 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.542 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-28T12:39:16.542 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.055913+0000', 'last_change': '2024-04-28T12:39:09.055913+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.055913+0000', 'last_undegraded': '2024-04-28T12:39:09.055913+0000', 'last_fullsized': '2024-04-28T12:39:09.055913+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-28T12:39:16.542 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-28T12:39:16.542 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.542 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-28T12:39:16.542 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.055649+0000', 'last_change': '2024-04-28T12:39:09.055649+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.055649+0000', 'last_undegraded': '2024-04-28T12:39:09.055649+0000', 'last_fullsized': '2024-04-28T12:39:09.055649+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-28T12:39:16.542 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-28T12:39:16.542 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.542 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-28T12:39:16.542 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.051406+0000', 'last_change': '2024-04-28T12:39:09.051406+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.051406+0000', 'last_undegraded': '2024-04-28T12:39:09.051406+0000', 'last_fullsized': '2024-04-28T12:39:09.051406+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-28T12:39:16.542 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-04-28T12:39:16.543 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.543 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-28T12:39:16.543 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.051769+0000', 'last_change': '2024-04-28T12:39:09.051769+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.051769+0000', 'last_undegraded': '2024-04-28T12:39:09.051769+0000', 'last_fullsized': '2024-04-28T12:39:09.051769+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-28T12:39:16.543 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-04-28T12:39:16.543 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.543 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-28T12:39:16.543 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.057585+0000', 'last_change': '2024-04-28T12:39:09.057585+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.057585+0000', 'last_undegraded': '2024-04-28T12:39:09.057585+0000', 'last_fullsized': '2024-04-28T12:39:09.057585+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-28T12:39:16.543 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-04-28T12:39:16.543 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-28T12:39:10.044250+0000', 'last_change': '2024-04-28T12:39:10.044250+0000', 'last_active': '2024-04-28T12:39:10.044250+0000', 'last_peered': '2024-04-28T12:39:10.044250+0000', 'last_clean': '2024-04-28T12:39:10.044250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:10.044250+0000', 'last_undegraded': '2024-04-28T12:39:10.044250+0000', 'last_fullsized': '2024-04-28T12:39:10.044250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:10.044250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-28T12:39:16.543 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-28T12:39:16.544 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 35, 'state': 'creating+peering', 'last_fresh': '2024-04-28T12:39:09.063490+0000', 'last_change': '2024-04-28T12:39:09.063490+0000', 'last_active': '2024-04-28T12:39:09.029756+0000', 'last_peered': '2024-04-28T12:39:09.029756+0000', 'last_clean': '2024-04-28T12:39:09.029756+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-28T12:39:09.063490+0000', 'last_undegraded': '2024-04-28T12:39:09.063490+0000', 'last_fullsized': '2024-04-28T12:39:09.063490+0000', 'mapping_epoch': 35, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 35, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'last_clean_scrub_stamp': '2024-04-28T12:39:09.029756+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-28T12:39:16.544 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-28T12:39:16.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.762+0000 7f1917200640 1 Processor -- start 2024-04-28T12:39:16.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.763+0000 7f1917200640 1 -- start start 2024-04-28T12:39:16.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.764+0000 7f1917200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f191815b900 0x7f19180b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:16.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.764+0000 7f1917200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f191806f840 0x7f191815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:16.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.764+0000 7f1917200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1918071d20 con 0x7f191815fce0 2024-04-28T12:39:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.764+0000 7f1917200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f1918160130 con 0x7f191815b900 2024-04-28T12:39:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.764+0000 7f1917200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f19180b4f00 con 0x7f191806f840 2024-04-28T12:39:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.765+0000 7f190fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f191806f840 0x7f191815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.765+0000 7f1915e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f191815b900 0x7f19180b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.765+0000 7f190fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f191806f840 0x7f191815f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:54768/0 (socket says 172.21.15.131:54768) 2024-04-28T12:39:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.765+0000 7f190fe00640 1 -- 172.21.15.131:0/1583104837 learned_addr learned my addr 172.21.15.131:0/1583104837 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:16.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.766+0000 7f1915e00640 1 -- 172.21.15.131:0/1583104837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f191806f840 msgr2=0x7f191815f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:16.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.766+0000 7f1915e00640 1 --2- 172.21.15.131:0/1583104837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f191806f840 0x7f191815f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.766+0000 7f1915e00640 1 -- 172.21.15.131:0/1583104837 >> v1:172.21.15.157:6789/0 conn(0x7f191815fce0 legacy=0x7f19180724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:16.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.766+0000 7f1915e00640 1 -- 172.21.15.131:0/1583104837 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19180b51e0 con 0x7f191815b900 2024-04-28T12:39:16.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.767+0000 7f1915e00640 1 --2- 172.21.15.131:0/1583104837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191815b900 0x7f19180b46d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f19080097a0 tx=0x7f190802c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d9a890a62cdc59e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:16.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.768+0000 7f1915400640 1 -- 172.21.15.131:0/1583104837 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908039070 con 0x7f191815b900 2024-04-28T12:39:16.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.768+0000 7f1915400640 1 -- 172.21.15.131:0/1583104837 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1908034640 con 0x7f191815b900 2024-04-28T12:39:16.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.768+0000 7f1915400640 1 -- 172.21.15.131:0/1583104837 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908034a10 con 0x7f191815b900 2024-04-28T12:39:16.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.768+0000 7f1917200640 1 -- 172.21.15.131:0/1583104837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191815b900 msgr2=0x7f19180b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:16.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.768+0000 7f1917200640 1 --2- 172.21.15.131:0/1583104837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191815b900 0x7f19180b46d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f19080097a0 tx=0x7f190802c390 comp rx=0 tx=0).stop 2024-04-28T12:39:16.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.769+0000 7f1917200640 1 -- 172.21.15.131:0/1583104837 shutdown_connections 2024-04-28T12:39:16.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.769+0000 7f1917200640 1 --2- 172.21.15.131:0/1583104837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f191806f840 0x7f191815f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.769+0000 7f1917200640 1 --2- 172.21.15.131:0/1583104837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191815b900 0x7f19180b46d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.769+0000 7f1917200640 1 -- 172.21.15.131:0/1583104837 >> 172.21.15.131:0/1583104837 conn(0x7f19180b26b0 msgr2=0x7f19180b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:16.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.769+0000 7f1917200640 1 -- 172.21.15.131:0/1583104837 shutdown_connections 2024-04-28T12:39:16.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.769+0000 7f1917200640 1 -- 172.21.15.131:0/1583104837 wait complete. 2024-04-28T12:39:16.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.770+0000 7f1917200640 1 Processor -- start 2024-04-28T12:39:16.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.771+0000 7f1917200640 1 -- start start 2024-04-28T12:39:16.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.771+0000 7f1917200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f191806f840 0x7f19180cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:16.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.771+0000 7f1917200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f191815b900 0x7f19181f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:16.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.771+0000 7f1917200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191815fce0 0x7f1918153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:16.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.771+0000 7f1917200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f19180b54c0 con 0x7f191815b900 2024-04-28T12:39:16.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f1917200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f19180b4c10 con 0x7f191815fce0 2024-04-28T12:39:16.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f1917200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f19180b5640 con 0x7f191806f840 2024-04-28T12:39:16.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f190fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f191815b900 0x7f19181f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:16.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f190fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f191815b900 0x7f19181f1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57406/0 (socket says 172.21.15.131:57406) 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f190fe00640 1 -- 172.21.15.131:0/534161944 learned_addr learned my addr 172.21.15.131:0/534161944 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f1915e00640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f191806f840 0x7f19180cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f1916800640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191815fce0 0x7f1918153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f190fe00640 1 -- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f191806f840 msgr2=0x7f19180cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f190fe00640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f191806f840 0x7f19180cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f190fe00640 1 -- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191815fce0 msgr2=0x7f1918153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f190fe00640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191815fce0 0x7f1918153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f190fe00640 1 -- 172.21.15.131:0/534161944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19180ce720 con 0x7f191815b900 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.772+0000 7f1916800640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191815fce0 0x7f1918153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.773+0000 7f190fe00640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f191815b900 0x7f19181f1790 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1900009fd0 tx=0x7f190000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.773+0000 7f190e000640 1 -- 172.21.15.131:0/534161944 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900017070 con 0x7f191815b900 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.773+0000 7f190e000640 1 -- 172.21.15.131:0/534161944 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f190000c940 con 0x7f191815b900 2024-04-28T12:39:16.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.773+0000 7f1917200640 1 -- 172.21.15.131:0/534161944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19180cea00 con 0x7f191815b900 2024-04-28T12:39:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.773+0000 7f190e000640 1 -- 172.21.15.131:0/534161944 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190000cd10 con 0x7f191815b900 2024-04-28T12:39:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.773+0000 7f1917200640 1 -- 172.21.15.131:0/534161944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191814f2f0 con 0x7f191815b900 2024-04-28T12:39:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.774+0000 7f1917200640 1 -- 172.21.15.131:0/534161944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1918069350 con 0x7f191815b900 2024-04-28T12:39:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.775+0000 7f190e000640 1 -- 172.21.15.131:0/534161944 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f190001a070 con 0x7f191815b900 2024-04-28T12:39:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.776+0000 7f190e000640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f18f406c8f0 0x7f18f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.776+0000 7f190e000640 1 -- 172.21.15.131:0/534161944 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f190008bfe0 con 0x7f191815b900 2024-04-28T12:39:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.777+0000 7f1915e00640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f18f406c8f0 0x7f18f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.778+0000 7f1915e00640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f18f406c8f0 0x7f18f406edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1908036000 tx=0x7f1908030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:16.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:16.781+0000 7f190e000640 1 -- 172.21.15.131:0/534161944 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f190005a270 con 0x7f191815b900 2024-04-28T12:39:17.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.050+0000 7f1917200640 1 -- 172.21.15.131:0/534161944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1918156270 con 0x7f191815b900 2024-04-28T12:39:17.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.050+0000 7f190e000640 1 -- 172.21.15.131:0/534161944 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1110 (secure 0 0 0) 0x7f1900059b70 con 0x7f191815b900 2024-04-28T12:39:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:17.047 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":134,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":38,"num_osds":8,"num_up_osds":8,"osd_up_since":1714307834,"num_in_osds":8,"osd_in_since":1714307826,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":242786304,"bytes_avail":772851326976,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"d","status":"up:active","gid":34610}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-04-28T12:38:26.940181+0000","services":{}},"progress_events":{}} 2024-04-28T12:39:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.053+0000 7f1917200640 1 -- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f18f406c8f0 msgr2=0x7f18f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.053+0000 7f1917200640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f18f406c8f0 0x7f18f406edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1908036000 tx=0x7f1908030000 comp rx=0 tx=0).stop 2024-04-28T12:39:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.053+0000 7f1917200640 1 -- 172.21.15.131:0/534161944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f191815b900 msgr2=0x7f19181f1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.053+0000 7f1917200640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f191815b900 0x7f19181f1790 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1900009fd0 tx=0x7f190000ed40 comp rx=0 tx=0).stop 2024-04-28T12:39:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.053+0000 7f1917200640 1 -- 172.21.15.131:0/534161944 shutdown_connections 2024-04-28T12:39:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.053+0000 7f1917200640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f18f406c8f0 0x7f18f406edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.053+0000 7f1917200640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191815fce0 0x7f1918153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.053+0000 7f1917200640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f191815b900 0x7f19181f1790 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.053+0000 7f1917200640 1 --2- 172.21.15.131:0/534161944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f191806f840 0x7f19180cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.053+0000 7f1917200640 1 -- 172.21.15.131:0/534161944 >> 172.21.15.131:0/534161944 conn(0x7f19180b26b0 msgr2=0x7f1918156a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:17.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.053+0000 7f1917200640 1 -- 172.21.15.131:0/534161944 shutdown_connections 2024-04-28T12:39:17.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.053+0000 7f1917200640 1 -- 172.21.15.131:0/534161944 wait complete. 2024-04-28T12:39:17.066 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-28T12:39:17.066 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:39:17.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.263+0000 7f8953e00640 1 Processor -- start 2024-04-28T12:39:17.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.263+0000 7f8953e00640 1 -- start start 2024-04-28T12:39:17.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.264+0000 7f8953e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8954072620 0x7f89540cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:17.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.264+0000 7f8953e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f89540ce2b0 0x7f895406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:17.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8953e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8954072a80 con 0x7f89540cd180 2024-04-28T12:39:17.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8953e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f89540cd560 con 0x7f89540ce2b0 2024-04-28T12:39:17.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8953e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8954072d00 con 0x7f8954072620 2024-04-28T12:39:17.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8952a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f89540cd180 0x7f895415dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38754/0 (socket says 172.21.15.131:38754) 2024-04-28T12:39:17.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8952a00640 1 -- 172.21.15.131:0/3019640942 learned_addr learned my addr 172.21.15.131:0/3019640942 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:17.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8953400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f89540ce2b0 0x7f895406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:17.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8953400640 1 -- 172.21.15.131:0/3019640942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8954072620 msgr2=0x7f89540cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:17.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8952000640 1 --2- 172.21.15.131:0/3019640942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8954072620 0x7f89540cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:17.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8953400640 1 --2- 172.21.15.131:0/3019640942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8954072620 0x7f89540cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8951600640 1 -- 172.21.15.131:0/3019640942 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3081245717 0 0) 0x7f8944001110 con 0x7f89540cd180 2024-04-28T12:39:17.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8953400640 1 -- 172.21.15.131:0/3019640942 >> v1:172.21.15.131:6789/0 conn(0x7f89540cd180 legacy=0x7f895415dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:17.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8953400640 1 -- 172.21.15.131:0/3019640942 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8954072fe0 con 0x7f89540ce2b0 2024-04-28T12:39:17.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.265+0000 7f8952000640 1 --2- 172.21.15.131:0/3019640942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8954072620 0x7f89540cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:17.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.266+0000 7f8953400640 1 --2- 172.21.15.131:0/3019640942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89540ce2b0 0x7f895406dac0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f894c009d90 tx=0x7f894c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62822023103e443 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:17.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.267+0000 7f8951600640 1 -- 172.21.15.131:0/3019640942 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c007740 con 0x7f89540ce2b0 2024-04-28T12:39:17.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.267+0000 7f8951600640 1 -- 172.21.15.131:0/3019640942 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f894c0078e0 con 0x7f89540ce2b0 2024-04-28T12:39:17.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.267+0000 7f8951600640 1 -- 172.21.15.131:0/3019640942 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c007cb0 con 0x7f89540ce2b0 2024-04-28T12:39:17.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.267+0000 7f8953e00640 1 -- 172.21.15.131:0/3019640942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89540ce2b0 msgr2=0x7f895406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:17.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.267+0000 7f8953e00640 1 --2- 172.21.15.131:0/3019640942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89540ce2b0 0x7f895406dac0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f894c009d90 tx=0x7f894c02a660 comp rx=0 tx=0).stop 2024-04-28T12:39:17.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.267+0000 7f8953e00640 1 -- 172.21.15.131:0/3019640942 shutdown_connections 2024-04-28T12:39:17.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.267+0000 7f8953e00640 1 --2- 172.21.15.131:0/3019640942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89540ce2b0 0x7f895406dac0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.267+0000 7f8953e00640 1 --2- 172.21.15.131:0/3019640942 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8954072620 0x7f89540cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.267+0000 7f8953e00640 1 -- 172.21.15.131:0/3019640942 >> 172.21.15.131:0/3019640942 conn(0x7f8954068090 msgr2=0x7f8954068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:17.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.267+0000 7f8953e00640 1 -- 172.21.15.131:0/3019640942 shutdown_connections 2024-04-28T12:39:17.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.268+0000 7f8953e00640 1 -- 172.21.15.131:0/3019640942 wait complete. 2024-04-28T12:39:17.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.269+0000 7f8953e00640 1 Processor -- start 2024-04-28T12:39:17.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.270+0000 7f8953e00640 1 -- start start 2024-04-28T12:39:17.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.271+0000 7f8953e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8954072620 0x7f89540b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:17.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.271+0000 7f8953e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89540cd180 0x7f89540aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:17.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.271+0000 7f8953e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89540ce2b0 0x7f89540c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:17.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.271+0000 7f8953e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8954073da0 con 0x7f89540ce2b0 2024-04-28T12:39:17.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.271+0000 7f8953e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89540735a0 con 0x7f89540cd180 2024-04-28T12:39:17.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.271+0000 7f8953e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f895406e000 con 0x7f8954072620 2024-04-28T12:39:17.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.271+0000 7f8953400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89540ce2b0 0x7f89540c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:17.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.271+0000 7f8953400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89540ce2b0 0x7f89540c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57428/0 (socket says 172.21.15.131:57428) 2024-04-28T12:39:17.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.271+0000 7f8953400640 1 -- 172.21.15.131:0/1200752189 learned_addr learned my addr 172.21.15.131:0/1200752189 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:17.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.271+0000 7f8952a00640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8954072620 0x7f89540b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:17.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.271+0000 7f8952000640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89540cd180 0x7f89540aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:17.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.272+0000 7f8953400640 1 -- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8954072620 msgr2=0x7f89540b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:17.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.272+0000 7f8953400640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8954072620 0x7f89540b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.272+0000 7f8953400640 1 -- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89540cd180 msgr2=0x7f89540aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:17.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.272+0000 7f8953400640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89540cd180 0x7f89540aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.272+0000 7f8953400640 1 -- 172.21.15.131:0/1200752189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89541f5370 con 0x7f89540ce2b0 2024-04-28T12:39:17.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.272+0000 7f8952000640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89540cd180 0x7f89540aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:17.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.272+0000 7f8953400640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89540ce2b0 0x7f89540c58b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f894c009eb0 tx=0x7f894c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:17.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.272+0000 7f894aa00640 1 -- 172.21.15.131:0/1200752189 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c03f070 con 0x7f89540ce2b0 2024-04-28T12:39:17.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.272+0000 7f894aa00640 1 -- 172.21.15.131:0/1200752189 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f894c031c90 con 0x7f89540ce2b0 2024-04-28T12:39:17.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.272+0000 7f894aa00640 1 -- 172.21.15.131:0/1200752189 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c044680 con 0x7f89540ce2b0 2024-04-28T12:39:17.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.272+0000 7f8953e00640 1 -- 172.21.15.131:0/1200752189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89541f5650 con 0x7f89540ce2b0 2024-04-28T12:39:17.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.273+0000 7f8953e00640 1 -- 172.21.15.131:0/1200752189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89541f5c10 con 0x7f89540ce2b0 2024-04-28T12:39:17.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.274+0000 7f8953e00640 1 -- 172.21.15.131:0/1200752189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8954069350 con 0x7f89540ce2b0 2024-04-28T12:39:17.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.275+0000 7f894aa00640 1 -- 172.21.15.131:0/1200752189 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f894c03b410 con 0x7f89540ce2b0 2024-04-28T12:39:17.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.276+0000 7f894aa00640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f893406c8f0 0x7f893406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:17.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.276+0000 7f894aa00640 1 -- 172.21.15.131:0/1200752189 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f894c0ad6a0 con 0x7f89540ce2b0 2024-04-28T12:39:17.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.276+0000 7f8952a00640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f893406c8f0 0x7f893406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:17.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.277+0000 7f8952a00640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f893406c8f0 0x7f893406edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8944000f60 tx=0x7f8944007970 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:17.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.280+0000 7f894aa00640 1 -- 172.21.15.131:0/1200752189 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f894c07b960 con 0x7f89540ce2b0 2024-04-28T12:39:17.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.476+0000 7f8953e00640 1 -- 172.21.15.131:0/1200752189 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f89540d0670 con 0x7f893406c8f0 2024-04-28T12:39:17.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.497+0000 7f894aa00640 1 -- 172.21.15.131:0/1200752189 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f89540d0670 con 0x7f893406c8f0 2024-04-28T12:39:17.497 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:17.499 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:39:17.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.503+0000 7f8953e00640 1 -- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f893406c8f0 msgr2=0x7f893406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:17.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.503+0000 7f8953e00640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f893406c8f0 0x7f893406edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8944000f60 tx=0x7f8944007970 comp rx=0 tx=0).stop 2024-04-28T12:39:17.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.503+0000 7f8953e00640 1 -- 172.21.15.131:0/1200752189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89540ce2b0 msgr2=0x7f89540c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:17.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.503+0000 7f8953e00640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89540ce2b0 0x7f89540c58b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f894c009eb0 tx=0x7f894c004a40 comp rx=0 tx=0).stop 2024-04-28T12:39:17.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.503+0000 7f8953e00640 1 -- 172.21.15.131:0/1200752189 shutdown_connections 2024-04-28T12:39:17.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.503+0000 7f8953e00640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f893406c8f0 0x7f893406edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.503+0000 7f8953e00640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89540ce2b0 0x7f89540c58b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.503+0000 7f8953e00640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89540cd180 0x7f89540aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.503+0000 7f8953e00640 1 --2- 172.21.15.131:0/1200752189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8954072620 0x7f89540b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:17.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.503+0000 7f8953e00640 1 -- 172.21.15.131:0/1200752189 >> 172.21.15.131:0/1200752189 conn(0x7f8954068090 msgr2=0x7f89540cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:17.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.504+0000 7f8953e00640 1 -- 172.21.15.131:0/1200752189 shutdown_connections 2024-04-28T12:39:17.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:17.504+0000 7f8953e00640 1 -- 172.21.15.131:0/1200752189 wait complete. 2024-04-28T12:39:17.518 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-04-28T12:39:16.959665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237492,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737228,"statfs":{"total":773094113280,"available":772850921472,"internally_reserved":0,"allocated":4759552,"data_stored":2705844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.896544"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511760+0000","last_change":"2024-04-28T12:39:11.103632+0000","last_active":"2024-04-28T12:39:12.511760+0000","last_peered":"2024-04-28T12:39:12.511760+0000","last_clean":"2024-04-28T12:39:12.511760+0000","last_became_active":"2024-04-28T12:39:11.102942+0000","last_became_peered":"2024-04-28T12:39:11.102942+0000","last_unstale":"2024-04-28T12:39:12.511760+0000","last_undegraded":"2024-04-28T12:39:12.511760+0000","last_fullsized":"2024-04-28T12:39:12.511760+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:42:35.965759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511257+0000","last_change":"2024-04-28T12:39:11.093817+0000","last_active":"2024-04-28T12:39:12.511257+0000","last_peered":"2024-04-28T12:39:12.511257+0000","last_clean":"2024-04-28T12:39:12.511257+0000","last_became_active":"2024-04-28T12:39:11.089057+0000","last_became_peered":"2024-04-28T12:39:11.089057+0000","last_unstale":"2024-04-28T12:39:12.511257+0000","last_undegraded":"2024-04-28T12:39:12.511257+0000","last_fullsized":"2024-04-28T12:39:12.511257+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:28:06.491386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105034+0000","last_change":"2024-04-28T12:39:11.099167+0000","last_active":"2024-04-28T12:39:13.105034+0000","last_peered":"2024-04-28T12:39:13.105034+0000","last_clean":"2024-04-28T12:39:13.105034+0000","last_became_active":"2024-04-28T12:39:11.098888+0000","last_became_peered":"2024-04-28T12:39:11.098888+0000","last_unstale":"2024-04-28T12:39:13.105034+0000","last_undegraded":"2024-04-28T12:39:13.105034+0000","last_fullsized":"2024-04-28T12:39:13.105034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:38:02.354280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094346+0000","last_change":"2024-04-28T12:39:11.082989+0000","last_active":"2024-04-28T12:39:12.094346+0000","last_peered":"2024-04-28T12:39:12.094346+0000","last_clean":"2024-04-28T12:39:12.094346+0000","last_became_active":"2024-04-28T12:39:11.082635+0000","last_became_peered":"2024-04-28T12:39:11.082635+0000","last_unstale":"2024-04-28T12:39:12.094346+0000","last_undegraded":"2024-04-28T12:39:12.094346+0000","last_fullsized":"2024-04-28T12:39:12.094346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:46:57.693015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093710+0000","last_change":"2024-04-28T12:39:11.082537+0000","last_active":"2024-04-28T12:39:12.093710+0000","last_peered":"2024-04-28T12:39:12.093710+0000","last_clean":"2024-04-28T12:39:12.093710+0000","last_became_active":"2024-04-28T12:39:11.082219+0000","last_became_peered":"2024-04-28T12:39:11.082219+0000","last_unstale":"2024-04-28T12:39:12.093710+0000","last_undegraded":"2024-04-28T12:39:12.093710+0000","last_fullsized":"2024-04-28T12:39:12.093710+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:50:52.412029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276957+0000","last_change":"2024-04-28T12:39:11.103589+0000","last_active":"2024-04-28T12:39:13.276957+0000","last_peered":"2024-04-28T12:39:13.276957+0000","last_clean":"2024-04-28T12:39:13.276957+0000","last_became_active":"2024-04-28T12:39:11.102204+0000","last_became_peered":"2024-04-28T12:39:11.102204+0000","last_unstale":"2024-04-28T12:39:13.276957+0000","last_undegraded":"2024-04-28T12:39:13.276957+0000","last_fullsized":"2024-04-28T12:39:13.276957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:30.942037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.355840+0000","last_change":"2024-04-28T12:39:11.106371+0000","last_active":"2024-04-28T12:39:12.355840+0000","last_peered":"2024-04-28T12:39:12.355840+0000","last_clean":"2024-04-28T12:39:12.355840+0000","last_became_active":"2024-04-28T12:39:11.106005+0000","last_became_peered":"2024-04-28T12:39:11.106005+0000","last_unstale":"2024-04-28T12:39:12.355840+0000","last_undegraded":"2024-04-28T12:39:12.355840+0000","last_fullsized":"2024-04-28T12:39:12.355840+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:27:11.732145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.510817+0000","last_change":"2024-04-28T12:39:11.108164+0000","last_active":"2024-04-28T12:39:12.510817+0000","last_peered":"2024-04-28T12:39:12.510817+0000","last_clean":"2024-04-28T12:39:12.510817+0000","last_became_active":"2024-04-28T12:39:11.107836+0000","last_became_peered":"2024-04-28T12:39:11.107836+0000","last_unstale":"2024-04-28T12:39:12.510817+0000","last_undegraded":"2024-04-28T12:39:12.510817+0000","last_fullsized":"2024-04-28T12:39:12.510817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:48:28.059974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105875+0000","last_change":"2024-04-28T12:39:11.105441+0000","last_active":"2024-04-28T12:39:13.105875+0000","last_peered":"2024-04-28T12:39:13.105875+0000","last_clean":"2024-04-28T12:39:13.105875+0000","last_became_active":"2024-04-28T12:39:11.105063+0000","last_became_peered":"2024-04-28T12:39:11.105063+0000","last_unstale":"2024-04-28T12:39:13.105875+0000","last_undegraded":"2024-04-28T12:39:13.105875+0000","last_fullsized":"2024-04-28T12:39:13.105875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:30.942037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.357011+0000","last_change":"2024-04-28T12:39:11.104694+0000","last_active":"2024-04-28T12:39:12.357011+0000","last_peered":"2024-04-28T12:39:12.357011+0000","last_clean":"2024-04-28T12:39:12.357011+0000","last_became_active":"2024-04-28T12:39:11.104406+0000","last_became_peered":"2024-04-28T12:39:11.104406+0000","last_unstale":"2024-04-28T12:39:12.357011+0000","last_undegraded":"2024-04-28T12:39:12.357011+0000","last_fullsized":"2024-04-28T12:39:12.357011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:41:28.559411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356910+0000","last_change":"2024-04-28T12:39:11.101140+0000","last_active":"2024-04-28T12:39:12.356910+0000","last_peered":"2024-04-28T12:39:12.356910+0000","last_clean":"2024-04-28T12:39:12.356910+0000","last_became_active":"2024-04-28T12:39:11.100193+0000","last_became_peered":"2024-04-28T12:39:11.100193+0000","last_unstale":"2024-04-28T12:39:12.356910+0000","last_undegraded":"2024-04-28T12:39:12.356910+0000","last_fullsized":"2024-04-28T12:39:12.356910+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:11:04.785848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105292+0000","last_change":"2024-04-28T12:39:11.090814+0000","last_active":"2024-04-28T12:39:13.105292+0000","last_peered":"2024-04-28T12:39:13.105292+0000","last_clean":"2024-04-28T12:39:13.105292+0000","last_became_active":"2024-04-28T12:39:11.090509+0000","last_became_peered":"2024-04-28T12:39:11.090509+0000","last_unstale":"2024-04-28T12:39:13.105292+0000","last_undegraded":"2024-04-28T12:39:13.105292+0000","last_fullsized":"2024-04-28T12:39:13.105292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:53:12.218233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356081+0000","last_change":"2024-04-28T12:39:11.110526+0000","last_active":"2024-04-28T12:39:12.356081+0000","last_peered":"2024-04-28T12:39:12.356081+0000","last_clean":"2024-04-28T12:39:12.356081+0000","last_became_active":"2024-04-28T12:39:11.110239+0000","last_became_peered":"2024-04-28T12:39:11.110239+0000","last_unstale":"2024-04-28T12:39:12.356081+0000","last_undegraded":"2024-04-28T12:39:12.356081+0000","last_fullsized":"2024-04-28T12:39:12.356081+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:46:13.934855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.096190+0000","last_change":"2024-04-28T12:39:11.090327+0000","last_active":"2024-04-28T12:39:12.096190+0000","last_peered":"2024-04-28T12:39:12.096190+0000","last_clean":"2024-04-28T12:39:12.096190+0000","last_became_active":"2024-04-28T12:39:11.090002+0000","last_became_peered":"2024-04-28T12:39:11.090002+0000","last_unstale":"2024-04-28T12:39:12.096190+0000","last_undegraded":"2024-04-28T12:39:12.096190+0000","last_fullsized":"2024-04-28T12:39:12.096190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:42.448610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356025+0000","last_change":"2024-04-28T12:39:11.100853+0000","last_active":"2024-04-28T12:39:12.356025+0000","last_peered":"2024-04-28T12:39:12.356025+0000","last_clean":"2024-04-28T12:39:12.356025+0000","last_became_active":"2024-04-28T12:39:11.100566+0000","last_became_peered":"2024-04-28T12:39:11.100566+0000","last_unstale":"2024-04-28T12:39:12.356025+0000","last_undegraded":"2024-04-28T12:39:12.356025+0000","last_fullsized":"2024-04-28T12:39:12.356025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:48:28.059974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276344+0000","last_change":"2024-04-28T12:39:11.113771+0000","last_active":"2024-04-28T12:39:13.276344+0000","last_peered":"2024-04-28T12:39:13.276344+0000","last_clean":"2024-04-28T12:39:13.276344+0000","last_became_active":"2024-04-28T12:39:11.112983+0000","last_became_peered":"2024-04-28T12:39:11.112983+0000","last_unstale":"2024-04-28T12:39:13.276344+0000","last_undegraded":"2024-04-28T12:39:13.276344+0000","last_fullsized":"2024-04-28T12:39:13.276344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:53:12.218233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.277645+0000","last_change":"2024-04-28T12:39:11.114601+0000","last_active":"2024-04-28T12:39:13.277645+0000","last_peered":"2024-04-28T12:39:13.277645+0000","last_clean":"2024-04-28T12:39:13.277645+0000","last_became_active":"2024-04-28T12:39:11.114305+0000","last_became_peered":"2024-04-28T12:39:11.114305+0000","last_unstale":"2024-04-28T12:39:13.277645+0000","last_undegraded":"2024-04-28T12:39:13.277645+0000","last_fullsized":"2024-04-28T12:39:13.277645+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:46:13.934855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.095543+0000","last_change":"2024-04-28T12:39:11.082583+0000","last_active":"2024-04-28T12:39:12.095543+0000","last_peered":"2024-04-28T12:39:12.095543+0000","last_clean":"2024-04-28T12:39:12.095543+0000","last_became_active":"2024-04-28T12:39:11.081714+0000","last_became_peered":"2024-04-28T12:39:11.081714+0000","last_unstale":"2024-04-28T12:39:12.095543+0000","last_undegraded":"2024-04-28T12:39:12.095543+0000","last_fullsized":"2024-04-28T12:39:12.095543+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:28:23.770671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094644+0000","last_change":"2024-04-28T12:39:11.095705+0000","last_active":"2024-04-28T12:39:12.094644+0000","last_peered":"2024-04-28T12:39:12.094644+0000","last_clean":"2024-04-28T12:39:12.094644+0000","last_became_active":"2024-04-28T12:39:11.095446+0000","last_became_peered":"2024-04-28T12:39:11.095446+0000","last_unstale":"2024-04-28T12:39:12.094644+0000","last_undegraded":"2024-04-28T12:39:12.094644+0000","last_fullsized":"2024-04-28T12:39:12.094644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:58:12.246960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093824+0000","last_change":"2024-04-28T12:39:11.080533+0000","last_active":"2024-04-28T12:39:12.093824+0000","last_peered":"2024-04-28T12:39:12.093824+0000","last_clean":"2024-04-28T12:39:12.093824+0000","last_became_active":"2024-04-28T12:39:11.080087+0000","last_became_peered":"2024-04-28T12:39:11.080087+0000","last_unstale":"2024-04-28T12:39:12.093824+0000","last_undegraded":"2024-04-28T12:39:12.093824+0000","last_fullsized":"2024-04-28T12:39:12.093824+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:58.590224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356636+0000","last_change":"2024-04-28T12:39:11.108814+0000","last_active":"2024-04-28T12:39:12.356636+0000","last_peered":"2024-04-28T12:39:12.356636+0000","last_clean":"2024-04-28T12:39:12.356636+0000","last_became_active":"2024-04-28T12:39:11.108471+0000","last_became_peered":"2024-04-28T12:39:11.108471+0000","last_unstale":"2024-04-28T12:39:12.356636+0000","last_undegraded":"2024-04-28T12:39:12.356636+0000","last_fullsized":"2024-04-28T12:39:12.356636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:28:06.491386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.097492+0000","last_change":"2024-04-28T12:39:11.085015+0000","last_active":"2024-04-28T12:39:12.097492+0000","last_peered":"2024-04-28T12:39:12.097492+0000","last_clean":"2024-04-28T12:39:12.097492+0000","last_became_active":"2024-04-28T12:39:11.084662+0000","last_became_peered":"2024-04-28T12:39:11.084662+0000","last_unstale":"2024-04-28T12:39:12.097492+0000","last_undegraded":"2024-04-28T12:39:12.097492+0000","last_fullsized":"2024-04-28T12:39:12.097492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:07:18.235619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276444+0000","last_change":"2024-04-28T12:39:11.113671+0000","last_active":"2024-04-28T12:39:13.276444+0000","last_peered":"2024-04-28T12:39:13.276444+0000","last_clean":"2024-04-28T12:39:13.276444+0000","last_became_active":"2024-04-28T12:39:11.113172+0000","last_became_peered":"2024-04-28T12:39:11.113172+0000","last_unstale":"2024-04-28T12:39:13.276444+0000","last_undegraded":"2024-04-28T12:39:13.276444+0000","last_fullsized":"2024-04-28T12:39:13.276444+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:18:19.496186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093764+0000","last_change":"2024-04-28T12:39:11.107323+0000","last_active":"2024-04-28T12:39:12.093764+0000","last_peered":"2024-04-28T12:39:12.093764+0000","last_clean":"2024-04-28T12:39:12.093764+0000","last_became_active":"2024-04-28T12:39:11.107046+0000","last_became_peered":"2024-04-28T12:39:11.107046+0000","last_unstale":"2024-04-28T12:39:12.093764+0000","last_undegraded":"2024-04-28T12:39:12.093764+0000","last_fullsized":"2024-04-28T12:39:12.093764+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:14.004417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.095506+0000","last_change":"2024-04-28T12:39:11.090831+0000","last_active":"2024-04-28T12:39:12.095506+0000","last_peered":"2024-04-28T12:39:12.095506+0000","last_clean":"2024-04-28T12:39:12.095506+0000","last_became_active":"2024-04-28T12:39:11.090536+0000","last_became_peered":"2024-04-28T12:39:11.090536+0000","last_unstale":"2024-04-28T12:39:12.095506+0000","last_undegraded":"2024-04-28T12:39:12.095506+0000","last_fullsized":"2024-04-28T12:39:12.095506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:47:42.830968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.277132+0000","last_change":"2024-04-28T12:39:11.108464+0000","last_active":"2024-04-28T12:39:13.277132+0000","last_peered":"2024-04-28T12:39:13.277132+0000","last_clean":"2024-04-28T12:39:13.277132+0000","last_became_active":"2024-04-28T12:39:11.108192+0000","last_became_peered":"2024-04-28T12:39:11.108192+0000","last_unstale":"2024-04-28T12:39:13.277132+0000","last_undegraded":"2024-04-28T12:39:13.277132+0000","last_fullsized":"2024-04-28T12:39:13.277132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:19:24.083380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094967+0000","last_change":"2024-04-28T12:39:11.083774+0000","last_active":"2024-04-28T12:39:12.094967+0000","last_peered":"2024-04-28T12:39:12.094967+0000","last_clean":"2024-04-28T12:39:12.094967+0000","last_became_active":"2024-04-28T12:39:11.083507+0000","last_became_peered":"2024-04-28T12:39:11.083507+0000","last_unstale":"2024-04-28T12:39:12.094967+0000","last_undegraded":"2024-04-28T12:39:12.094967+0000","last_fullsized":"2024-04-28T12:39:12.094967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:42.448610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356798+0000","last_change":"2024-04-28T12:39:11.106902+0000","last_active":"2024-04-28T12:39:12.356798+0000","last_peered":"2024-04-28T12:39:12.356798+0000","last_clean":"2024-04-28T12:39:12.356798+0000","last_became_active":"2024-04-28T12:39:11.106618+0000","last_became_peered":"2024-04-28T12:39:11.106618+0000","last_unstale":"2024-04-28T12:39:12.356798+0000","last_undegraded":"2024-04-28T12:39:12.356798+0000","last_fullsized":"2024-04-28T12:39:12.356798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:39:14.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093895+0000","last_change":"2024-04-28T12:39:11.092014+0000","last_active":"2024-04-28T12:39:12.093895+0000","last_peered":"2024-04-28T12:39:12.093895+0000","last_clean":"2024-04-28T12:39:12.093895+0000","last_became_active":"2024-04-28T12:39:11.091756+0000","last_became_peered":"2024-04-28T12:39:11.091756+0000","last_unstale":"2024-04-28T12:39:12.093895+0000","last_undegraded":"2024-04-28T12:39:12.093895+0000","last_fullsized":"2024-04-28T12:39:12.093895+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:17.650325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094109+0000","last_change":"2024-04-28T12:39:11.091584+0000","last_active":"2024-04-28T12:39:12.094109+0000","last_peered":"2024-04-28T12:39:12.094109+0000","last_clean":"2024-04-28T12:39:12.094109+0000","last_became_active":"2024-04-28T12:39:11.091313+0000","last_became_peered":"2024-04-28T12:39:11.091313+0000","last_unstale":"2024-04-28T12:39:12.094109+0000","last_undegraded":"2024-04-28T12:39:12.094109+0000","last_fullsized":"2024-04-28T12:39:12.094109+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:13.938098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276580+0000","last_change":"2024-04-28T12:39:11.114417+0000","last_active":"2024-04-28T12:39:13.276580+0000","last_peered":"2024-04-28T12:39:13.276580+0000","last_clean":"2024-04-28T12:39:13.276580+0000","last_became_active":"2024-04-28T12:39:11.113295+0000","last_became_peered":"2024-04-28T12:39:11.113295+0000","last_unstale":"2024-04-28T12:39:13.276580+0000","last_undegraded":"2024-04-28T12:39:13.276580+0000","last_fullsized":"2024-04-28T12:39:13.276580+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:19:24.083380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276447+0000","last_change":"2024-04-28T12:39:11.088429+0000","last_active":"2024-04-28T12:39:13.276447+0000","last_peered":"2024-04-28T12:39:13.276447+0000","last_clean":"2024-04-28T12:39:13.276447+0000","last_became_active":"2024-04-28T12:39:11.088135+0000","last_became_peered":"2024-04-28T12:39:11.088135+0000","last_unstale":"2024-04-28T12:39:13.276447+0000","last_undegraded":"2024-04-28T12:39:13.276447+0000","last_fullsized":"2024-04-28T12:39:13.276447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:48:38.113586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276988+0000","last_change":"2024-04-28T12:39:11.115910+0000","last_active":"2024-04-28T12:39:13.276988+0000","last_peered":"2024-04-28T12:39:13.276988+0000","last_clean":"2024-04-28T12:39:13.276988+0000","last_became_active":"2024-04-28T12:39:11.115648+0000","last_became_peered":"2024-04-28T12:39:11.115648+0000","last_unstale":"2024-04-28T12:39:13.276988+0000","last_undegraded":"2024-04-28T12:39:13.276988+0000","last_fullsized":"2024-04-28T12:39:13.276988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:27:11.732145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.095650+0000","last_change":"2024-04-28T12:39:10.095609+0000","last_active":"2024-04-28T12:39:12.095650+0000","last_peered":"2024-04-28T12:39:12.095650+0000","last_clean":"2024-04-28T12:39:12.095650+0000","last_became_active":"2024-04-28T12:39:10.092425+0000","last_became_peered":"2024-04-28T12:39:10.092425+0000","last_unstale":"2024-04-28T12:39:12.095650+0000","last_undegraded":"2024-04-28T12:39:12.095650+0000","last_fullsized":"2024-04-28T12:39:12.095650+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:59:52.609458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276621+0000","last_change":"2024-04-28T12:39:11.109368+0000","last_active":"2024-04-28T12:39:13.276621+0000","last_peered":"2024-04-28T12:39:13.276621+0000","last_clean":"2024-04-28T12:39:13.276621+0000","last_became_active":"2024-04-28T12:39:11.109010+0000","last_became_peered":"2024-04-28T12:39:11.109010+0000","last_unstale":"2024-04-28T12:39:13.276621+0000","last_undegraded":"2024-04-28T12:39:13.276621+0000","last_fullsized":"2024-04-28T12:39:13.276621+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:53:12.218233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105370+0000","last_change":"2024-04-28T12:39:10.091734+0000","last_active":"2024-04-28T12:39:13.105370+0000","last_peered":"2024-04-28T12:39:13.105370+0000","last_clean":"2024-04-28T12:39:13.105370+0000","last_became_active":"2024-04-28T12:39:10.091472+0000","last_became_peered":"2024-04-28T12:39:10.091472+0000","last_unstale":"2024-04-28T12:39:13.105370+0000","last_undegraded":"2024-04-28T12:39:13.105370+0000","last_fullsized":"2024-04-28T12:39:13.105370+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:31:06.775995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.097917+0000","last_change":"2024-04-28T12:39:11.088057+0000","last_active":"2024-04-28T12:39:12.097917+0000","last_peered":"2024-04-28T12:39:12.097917+0000","last_clean":"2024-04-28T12:39:12.097917+0000","last_became_active":"2024-04-28T12:39:11.087786+0000","last_became_peered":"2024-04-28T12:39:11.087786+0000","last_unstale":"2024-04-28T12:39:12.097917+0000","last_undegraded":"2024-04-28T12:39:12.097917+0000","last_fullsized":"2024-04-28T12:39:12.097917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:37:11.157161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276852+0000","last_change":"2024-04-28T12:39:10.086242+0000","last_active":"2024-04-28T12:39:13.276852+0000","last_peered":"2024-04-28T12:39:13.276852+0000","last_clean":"2024-04-28T12:39:13.276852+0000","last_became_active":"2024-04-28T12:39:10.085476+0000","last_became_peered":"2024-04-28T12:39:10.085476+0000","last_unstale":"2024-04-28T12:39:13.276852+0000","last_undegraded":"2024-04-28T12:39:13.276852+0000","last_fullsized":"2024-04-28T12:39:13.276852+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:41.434116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511762+0000","last_change":"2024-04-28T12:39:11.104584+0000","last_active":"2024-04-28T12:39:12.511762+0000","last_peered":"2024-04-28T12:39:12.511762+0000","last_clean":"2024-04-28T12:39:12.511762+0000","last_became_active":"2024-04-28T12:39:11.103351+0000","last_became_peered":"2024-04-28T12:39:11.103351+0000","last_unstale":"2024-04-28T12:39:12.511762+0000","last_undegraded":"2024-04-28T12:39:12.511762+0000","last_fullsized":"2024-04-28T12:39:12.511762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:41:28.559411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.097159+0000","last_change":"2024-04-28T12:39:10.088525+0000","last_active":"2024-04-28T12:39:12.097159+0000","last_peered":"2024-04-28T12:39:12.097159+0000","last_clean":"2024-04-28T12:39:12.097159+0000","last_became_active":"2024-04-28T12:39:10.088154+0000","last_became_peered":"2024-04-28T12:39:10.088154+0000","last_unstale":"2024-04-28T12:39:12.097159+0000","last_undegraded":"2024-04-28T12:39:12.097159+0000","last_fullsized":"2024-04-28T12:39:12.097159+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:32:09.645564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.104651+0000","last_change":"2024-04-28T12:39:11.094511+0000","last_active":"2024-04-28T12:39:13.104651+0000","last_peered":"2024-04-28T12:39:13.104651+0000","last_clean":"2024-04-28T12:39:13.104651+0000","last_became_active":"2024-04-28T12:39:11.094246+0000","last_became_peered":"2024-04-28T12:39:11.094246+0000","last_unstale":"2024-04-28T12:39:13.104651+0000","last_undegraded":"2024-04-28T12:39:13.104651+0000","last_fullsized":"2024-04-28T12:39:13.104651+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:22:48.363451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356260+0000","last_change":"2024-04-28T12:39:10.096517+0000","last_active":"2024-04-28T12:39:12.356260+0000","last_peered":"2024-04-28T12:39:12.356260+0000","last_clean":"2024-04-28T12:39:12.356260+0000","last_became_active":"2024-04-28T12:39:10.096237+0000","last_became_peered":"2024-04-28T12:39:10.096237+0000","last_unstale":"2024-04-28T12:39:12.356260+0000","last_undegraded":"2024-04-28T12:39:12.356260+0000","last_fullsized":"2024-04-28T12:39:12.356260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:16.635831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356315+0000","last_change":"2024-04-28T12:39:11.110905+0000","last_active":"2024-04-28T12:39:12.356315+0000","last_peered":"2024-04-28T12:39:12.356315+0000","last_clean":"2024-04-28T12:39:12.356315+0000","last_became_active":"2024-04-28T12:39:11.110618+0000","last_became_peered":"2024-04-28T12:39:11.110618+0000","last_unstale":"2024-04-28T12:39:12.356315+0000","last_undegraded":"2024-04-28T12:39:12.356315+0000","last_fullsized":"2024-04-28T12:39:12.356315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:07.039610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276058+0000","last_change":"2024-04-28T12:39:10.100906+0000","last_active":"2024-04-28T12:39:13.276058+0000","last_peered":"2024-04-28T12:39:13.276058+0000","last_clean":"2024-04-28T12:39:13.276058+0000","last_became_active":"2024-04-28T12:39:10.100163+0000","last_became_peered":"2024-04-28T12:39:10.100163+0000","last_unstale":"2024-04-28T12:39:13.276058+0000","last_undegraded":"2024-04-28T12:39:13.276058+0000","last_fullsized":"2024-04-28T12:39:13.276058+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:16.635831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.096809+0000","last_change":"2024-04-28T12:39:11.099558+0000","last_active":"2024-04-28T12:39:12.096809+0000","last_peered":"2024-04-28T12:39:12.096809+0000","last_clean":"2024-04-28T12:39:12.096809+0000","last_became_active":"2024-04-28T12:39:11.097507+0000","last_became_peered":"2024-04-28T12:39:11.097507+0000","last_unstale":"2024-04-28T12:39:12.096809+0000","last_undegraded":"2024-04-28T12:39:12.096809+0000","last_fullsized":"2024-04-28T12:39:12.096809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:28:23.770671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276045+0000","last_change":"2024-04-28T12:39:10.102757+0000","last_active":"2024-04-28T12:39:13.276045+0000","last_peered":"2024-04-28T12:39:13.276045+0000","last_clean":"2024-04-28T12:39:13.276045+0000","last_became_active":"2024-04-28T12:39:10.102457+0000","last_became_peered":"2024-04-28T12:39:10.102457+0000","last_unstale":"2024-04-28T12:39:13.276045+0000","last_undegraded":"2024-04-28T12:39:13.276045+0000","last_fullsized":"2024-04-28T12:39:13.276045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:16.635831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.510901+0000","last_change":"2024-04-28T12:39:11.107070+0000","last_active":"2024-04-28T12:39:12.510901+0000","last_peered":"2024-04-28T12:39:12.510901+0000","last_clean":"2024-04-28T12:39:12.510901+0000","last_became_active":"2024-04-28T12:39:11.106734+0000","last_became_peered":"2024-04-28T12:39:11.106734+0000","last_unstale":"2024-04-28T12:39:12.510901+0000","last_undegraded":"2024-04-28T12:39:12.510901+0000","last_fullsized":"2024-04-28T12:39:12.510901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:46:13.934855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094998+0000","last_change":"2024-04-28T12:39:10.091842+0000","last_active":"2024-04-28T12:39:12.094998+0000","last_peered":"2024-04-28T12:39:12.094998+0000","last_clean":"2024-04-28T12:39:12.094998+0000","last_became_active":"2024-04-28T12:39:10.091502+0000","last_became_peered":"2024-04-28T12:39:10.091502+0000","last_unstale":"2024-04-28T12:39:12.094998+0000","last_undegraded":"2024-04-28T12:39:12.094998+0000","last_fullsized":"2024-04-28T12:39:12.094998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:48.009392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094560+0000","last_change":"2024-04-28T12:39:11.082913+0000","last_active":"2024-04-28T12:39:12.094560+0000","last_peered":"2024-04-28T12:39:12.094560+0000","last_clean":"2024-04-28T12:39:12.094560+0000","last_became_active":"2024-04-28T12:39:11.081913+0000","last_became_peered":"2024-04-28T12:39:11.081913+0000","last_unstale":"2024-04-28T12:39:12.094560+0000","last_undegraded":"2024-04-28T12:39:12.094560+0000","last_fullsized":"2024-04-28T12:39:12.094560+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:57:36.240279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511835+0000","last_change":"2024-04-28T12:39:10.080378+0000","last_active":"2024-04-28T12:39:12.511835+0000","last_peered":"2024-04-28T12:39:12.511835+0000","last_clean":"2024-04-28T12:39:12.511835+0000","last_became_active":"2024-04-28T12:39:10.080103+0000","last_became_peered":"2024-04-28T12:39:10.080103+0000","last_unstale":"2024-04-28T12:39:12.511835+0000","last_undegraded":"2024-04-28T12:39:12.511835+0000","last_fullsized":"2024-04-28T12:39:12.511835+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:57:30.849560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276902+0000","last_change":"2024-04-28T12:39:11.109905+0000","last_active":"2024-04-28T12:39:13.276902+0000","last_peered":"2024-04-28T12:39:13.276902+0000","last_clean":"2024-04-28T12:39:13.276902+0000","last_became_active":"2024-04-28T12:39:11.109423+0000","last_became_peered":"2024-04-28T12:39:11.109423+0000","last_unstale":"2024-04-28T12:39:13.276902+0000","last_undegraded":"2024-04-28T12:39:13.276902+0000","last_fullsized":"2024-04-28T12:39:13.276902+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:37:34.902184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.357268+0000","last_change":"2024-04-28T12:39:10.095840+0000","last_active":"2024-04-28T12:39:12.357268+0000","last_peered":"2024-04-28T12:39:12.357268+0000","last_clean":"2024-04-28T12:39:12.357268+0000","last_became_active":"2024-04-28T12:39:10.095551+0000","last_became_peered":"2024-04-28T12:39:10.095551+0000","last_unstale":"2024-04-28T12:39:12.357268+0000","last_undegraded":"2024-04-28T12:39:12.357268+0000","last_fullsized":"2024-04-28T12:39:12.357268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:47:41.816474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105575+0000","last_change":"2024-04-28T12:39:11.102650+0000","last_active":"2024-04-28T12:39:13.105575+0000","last_peered":"2024-04-28T12:39:13.105575+0000","last_clean":"2024-04-28T12:39:13.105575+0000","last_became_active":"2024-04-28T12:39:11.102392+0000","last_became_peered":"2024-04-28T12:39:11.102392+0000","last_unstale":"2024-04-28T12:39:13.105575+0000","last_undegraded":"2024-04-28T12:39:13.105575+0000","last_fullsized":"2024-04-28T12:39:13.105575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:18:19.496186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.357068+0000","last_change":"2024-04-28T12:39:10.098882+0000","last_active":"2024-04-28T12:39:12.357068+0000","last_peered":"2024-04-28T12:39:12.357068+0000","last_clean":"2024-04-28T12:39:12.357068+0000","last_became_active":"2024-04-28T12:39:10.098533+0000","last_became_peered":"2024-04-28T12:39:10.098533+0000","last_unstale":"2024-04-28T12:39:12.357068+0000","last_undegraded":"2024-04-28T12:39:12.357068+0000","last_fullsized":"2024-04-28T12:39:12.357068+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:28:22.756177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.097931+0000","last_change":"2024-04-28T12:39:11.086995+0000","last_active":"2024-04-28T12:39:12.097931+0000","last_peered":"2024-04-28T12:39:12.097931+0000","last_clean":"2024-04-28T12:39:12.097931+0000","last_became_active":"2024-04-28T12:39:11.086470+0000","last_became_peered":"2024-04-28T12:39:11.086470+0000","last_unstale":"2024-04-28T12:39:12.097931+0000","last_undegraded":"2024-04-28T12:39:12.097931+0000","last_fullsized":"2024-04-28T12:39:12.097931+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:22:48.363451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511090+0000","last_change":"2024-04-28T12:39:10.088368+0000","last_active":"2024-04-28T12:39:12.511090+0000","last_peered":"2024-04-28T12:39:12.511090+0000","last_clean":"2024-04-28T12:39:12.511090+0000","last_became_active":"2024-04-28T12:39:10.087872+0000","last_became_peered":"2024-04-28T12:39:10.087872+0000","last_unstale":"2024-04-28T12:39:12.511090+0000","last_undegraded":"2024-04-28T12:39:12.511090+0000","last_fullsized":"2024-04-28T12:39:12.511090+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:28:22.756177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.104697+0000","last_change":"2024-04-28T12:39:11.103093+0000","last_active":"2024-04-28T12:39:13.104697+0000","last_peered":"2024-04-28T12:39:13.104697+0000","last_clean":"2024-04-28T12:39:13.104697+0000","last_became_active":"2024-04-28T12:39:11.102769+0000","last_became_peered":"2024-04-28T12:39:11.102769+0000","last_unstale":"2024-04-28T12:39:13.104697+0000","last_undegraded":"2024-04-28T12:39:13.104697+0000","last_fullsized":"2024-04-28T12:39:13.104697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:56:39.106313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511218+0000","last_change":"2024-04-28T12:39:10.078842+0000","last_active":"2024-04-28T12:39:12.511218+0000","last_peered":"2024-04-28T12:39:12.511218+0000","last_clean":"2024-04-28T12:39:12.511218+0000","last_became_active":"2024-04-28T12:39:10.078571+0000","last_became_peered":"2024-04-28T12:39:10.078571+0000","last_unstale":"2024-04-28T12:39:12.511218+0000","last_undegraded":"2024-04-28T12:39:12.511218+0000","last_fullsized":"2024-04-28T12:39:12.511218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:37:10.142667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.510809+0000","last_change":"2024-04-28T12:39:11.096983+0000","last_active":"2024-04-28T12:39:12.510809+0000","last_peered":"2024-04-28T12:39:12.510809+0000","last_clean":"2024-04-28T12:39:12.510809+0000","last_became_active":"2024-04-28T12:39:11.096674+0000","last_became_peered":"2024-04-28T12:39:11.096674+0000","last_unstale":"2024-04-28T12:39:12.510809+0000","last_undegraded":"2024-04-28T12:39:12.510809+0000","last_fullsized":"2024-04-28T12:39:12.510809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:22.911705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276402+0000","last_change":"2024-04-28T12:39:10.092734+0000","last_active":"2024-04-28T12:39:13.276402+0000","last_peered":"2024-04-28T12:39:13.276402+0000","last_clean":"2024-04-28T12:39:13.276402+0000","last_became_active":"2024-04-28T12:39:10.092410+0000","last_became_peered":"2024-04-28T12:39:10.092410+0000","last_unstale":"2024-04-28T12:39:13.276402+0000","last_undegraded":"2024-04-28T12:39:13.276402+0000","last_fullsized":"2024-04-28T12:39:13.276402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:12.923604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276436+0000","last_change":"2024-04-28T12:39:11.112853+0000","last_active":"2024-04-28T12:39:13.276436+0000","last_peered":"2024-04-28T12:39:13.276436+0000","last_clean":"2024-04-28T12:39:13.276436+0000","last_became_active":"2024-04-28T12:39:11.112549+0000","last_became_peered":"2024-04-28T12:39:11.112549+0000","last_unstale":"2024-04-28T12:39:13.276436+0000","last_undegraded":"2024-04-28T12:39:13.276436+0000","last_fullsized":"2024-04-28T12:39:13.276436+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:30.942037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094048+0000","last_change":"2024-04-28T12:39:10.089193+0000","last_active":"2024-04-28T12:39:12.094048+0000","last_peered":"2024-04-28T12:39:12.094048+0000","last_clean":"2024-04-28T12:39:12.094048+0000","last_became_active":"2024-04-28T12:39:10.088837+0000","last_became_peered":"2024-04-28T12:39:10.088837+0000","last_unstale":"2024-04-28T12:39:12.094048+0000","last_undegraded":"2024-04-28T12:39:12.094048+0000","last_fullsized":"2024-04-28T12:39:12.094048+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:58:11.232466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511072+0000","last_change":"2024-04-28T12:39:11.098372+0000","last_active":"2024-04-28T12:39:12.511072+0000","last_peered":"2024-04-28T12:39:12.511072+0000","last_clean":"2024-04-28T12:39:12.511072+0000","last_became_active":"2024-04-28T12:39:11.098095+0000","last_became_peered":"2024-04-28T12:39:11.098095+0000","last_unstale":"2024-04-28T12:39:12.511072+0000","last_undegraded":"2024-04-28T12:39:12.511072+0000","last_fullsized":"2024-04-28T12:39:12.511072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:07.039610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276240+0000","last_change":"2024-04-28T12:39:10.100390+0000","last_active":"2024-04-28T12:39:13.276240+0000","last_peered":"2024-04-28T12:39:13.276240+0000","last_clean":"2024-04-28T12:39:13.276240+0000","last_became_active":"2024-04-28T12:39:10.100008+0000","last_became_peered":"2024-04-28T12:39:10.100008+0000","last_unstale":"2024-04-28T12:39:13.276240+0000","last_undegraded":"2024-04-28T12:39:13.276240+0000","last_fullsized":"2024-04-28T12:39:13.276240+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:57.575730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511653+0000","last_change":"2024-04-28T12:39:11.099956+0000","last_active":"2024-04-28T12:39:12.511653+0000","last_peered":"2024-04-28T12:39:12.511653+0000","last_clean":"2024-04-28T12:39:12.511653+0000","last_became_active":"2024-04-28T12:39:11.099709+0000","last_became_peered":"2024-04-28T12:39:11.099709+0000","last_unstale":"2024-04-28T12:39:12.511653+0000","last_undegraded":"2024-04-28T12:39:12.511653+0000","last_fullsized":"2024-04-28T12:39:12.511653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:07:42.536830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356803+0000","last_change":"2024-04-28T12:39:10.094644+0000","last_active":"2024-04-28T12:39:12.356803+0000","last_peered":"2024-04-28T12:39:12.356803+0000","last_clean":"2024-04-28T12:39:12.356803+0000","last_became_active":"2024-04-28T12:39:10.093807+0000","last_became_peered":"2024-04-28T12:39:10.093807+0000","last_unstale":"2024-04-28T12:39:12.356803+0000","last_undegraded":"2024-04-28T12:39:12.356803+0000","last_fullsized":"2024-04-28T12:39:12.356803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:12.923604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.357070+0000","last_change":"2024-04-28T12:39:11.112724+0000","last_active":"2024-04-28T12:39:12.357070+0000","last_peered":"2024-04-28T12:39:12.357070+0000","last_clean":"2024-04-28T12:39:12.357070+0000","last_became_active":"2024-04-28T12:39:11.112245+0000","last_became_peered":"2024-04-28T12:39:11.112245+0000","last_unstale":"2024-04-28T12:39:12.357070+0000","last_undegraded":"2024-04-28T12:39:12.357070+0000","last_fullsized":"2024-04-28T12:39:12.357070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:42:35.965759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.097939+0000","last_change":"2024-04-28T12:39:10.098183+0000","last_active":"2024-04-28T12:39:12.097939+0000","last_peered":"2024-04-28T12:39:12.097939+0000","last_clean":"2024-04-28T12:39:12.097939+0000","last_became_active":"2024-04-28T12:39:10.097444+0000","last_became_peered":"2024-04-28T12:39:10.097444+0000","last_unstale":"2024-04-28T12:39:12.097939+0000","last_undegraded":"2024-04-28T12:39:12.097939+0000","last_fullsized":"2024-04-28T12:39:12.097939+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:59:52.609458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.097788+0000","last_change":"2024-04-28T12:39:11.103256+0000","last_active":"2024-04-28T12:39:12.097788+0000","last_peered":"2024-04-28T12:39:12.097788+0000","last_clean":"2024-04-28T12:39:12.097788+0000","last_became_active":"2024-04-28T12:39:11.102960+0000","last_became_peered":"2024-04-28T12:39:11.102960+0000","last_unstale":"2024-04-28T12:39:12.097788+0000","last_undegraded":"2024-04-28T12:39:12.097788+0000","last_fullsized":"2024-04-28T12:39:12.097788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:57:31.864054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094781+0000","last_change":"2024-04-28T12:39:10.097191+0000","last_active":"2024-04-28T12:39:12.094781+0000","last_peered":"2024-04-28T12:39:12.094781+0000","last_clean":"2024-04-28T12:39:12.094781+0000","last_became_active":"2024-04-28T12:39:10.096896+0000","last_became_peered":"2024-04-28T12:39:10.096896+0000","last_unstale":"2024-04-28T12:39:12.094781+0000","last_undegraded":"2024-04-28T12:39:12.094781+0000","last_fullsized":"2024-04-28T12:39:12.094781+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:54:39.157609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094576+0000","last_change":"2024-04-28T12:39:11.099144+0000","last_active":"2024-04-28T12:39:12.094576+0000","last_peered":"2024-04-28T12:39:12.094576+0000","last_clean":"2024-04-28T12:39:12.094576+0000","last_became_active":"2024-04-28T12:39:11.098133+0000","last_became_peered":"2024-04-28T12:39:11.098133+0000","last_unstale":"2024-04-28T12:39:12.094576+0000","last_undegraded":"2024-04-28T12:39:12.094576+0000","last_fullsized":"2024-04-28T12:39:12.094576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:58.053771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276458+0000","last_change":"2024-04-28T12:39:10.085794+0000","last_active":"2024-04-28T12:39:13.276458+0000","last_peered":"2024-04-28T12:39:13.276458+0000","last_clean":"2024-04-28T12:39:13.276458+0000","last_became_active":"2024-04-28T12:39:10.085462+0000","last_became_peered":"2024-04-28T12:39:10.085462+0000","last_unstale":"2024-04-28T12:39:13.276458+0000","last_undegraded":"2024-04-28T12:39:13.276458+0000","last_fullsized":"2024-04-28T12:39:13.276458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:57.575730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276847+0000","last_change":"2024-04-28T12:39:11.113293+0000","last_active":"2024-04-28T12:39:13.276847+0000","last_peered":"2024-04-28T12:39:13.276847+0000","last_clean":"2024-04-28T12:39:13.276847+0000","last_became_active":"2024-04-28T12:39:11.112505+0000","last_became_peered":"2024-04-28T12:39:11.112505+0000","last_unstale":"2024-04-28T12:39:13.276847+0000","last_undegraded":"2024-04-28T12:39:13.276847+0000","last_fullsized":"2024-04-28T12:39:13.276847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:30:01.863945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093761+0000","last_change":"2024-04-28T12:39:10.080044+0000","last_active":"2024-04-28T12:39:12.093761+0000","last_peered":"2024-04-28T12:39:12.093761+0000","last_clean":"2024-04-28T12:39:12.093761+0000","last_became_active":"2024-04-28T12:39:10.079684+0000","last_became_peered":"2024-04-28T12:39:10.079684+0000","last_unstale":"2024-04-28T12:39:12.093761+0000","last_undegraded":"2024-04-28T12:39:12.093761+0000","last_fullsized":"2024-04-28T12:39:12.093761+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:47:56.416266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.104634+0000","last_change":"2024-04-28T12:39:11.106347+0000","last_active":"2024-04-28T12:39:13.104634+0000","last_peered":"2024-04-28T12:39:13.104634+0000","last_clean":"2024-04-28T12:39:13.104634+0000","last_became_active":"2024-04-28T12:39:11.106062+0000","last_became_peered":"2024-04-28T12:39:11.106062+0000","last_unstale":"2024-04-28T12:39:13.104634+0000","last_undegraded":"2024-04-28T12:39:13.104634+0000","last_fullsized":"2024-04-28T12:39:13.104634+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:37:11.157161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093886+0000","last_change":"2024-04-28T12:39:10.084948+0000","last_active":"2024-04-28T12:39:12.093886+0000","last_peered":"2024-04-28T12:39:12.093886+0000","last_clean":"2024-04-28T12:39:12.093886+0000","last_became_active":"2024-04-28T12:39:10.084658+0000","last_became_peered":"2024-04-28T12:39:10.084658+0000","last_unstale":"2024-04-28T12:39:12.093886+0000","last_undegraded":"2024-04-28T12:39:12.093886+0000","last_fullsized":"2024-04-28T12:39:12.093886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:48:58.062822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356067+0000","last_change":"2024-04-28T12:39:11.100927+0000","last_active":"2024-04-28T12:39:12.356067+0000","last_peered":"2024-04-28T12:39:12.356067+0000","last_clean":"2024-04-28T12:39:12.356067+0000","last_became_active":"2024-04-28T12:39:11.097873+0000","last_became_peered":"2024-04-28T12:39:11.097873+0000","last_unstale":"2024-04-28T12:39:12.356067+0000","last_undegraded":"2024-04-28T12:39:12.356067+0000","last_fullsized":"2024-04-28T12:39:12.356067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:22.911705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276599+0000","last_change":"2024-04-28T12:39:10.100038+0000","last_active":"2024-04-28T12:39:13.276599+0000","last_peered":"2024-04-28T12:39:13.276599+0000","last_clean":"2024-04-28T12:39:13.276599+0000","last_became_active":"2024-04-28T12:39:10.098896+0000","last_became_peered":"2024-04-28T12:39:10.098896+0000","last_unstale":"2024-04-28T12:39:13.276599+0000","last_undegraded":"2024-04-28T12:39:13.276599+0000","last_fullsized":"2024-04-28T12:39:13.276599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:06:28.119829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511320+0000","last_change":"2024-04-28T12:39:11.105099+0000","last_active":"2024-04-28T12:39:12.511320+0000","last_peered":"2024-04-28T12:39:12.511320+0000","last_clean":"2024-04-28T12:39:12.511320+0000","last_became_active":"2024-04-28T12:39:11.104777+0000","last_became_peered":"2024-04-28T12:39:11.104777+0000","last_unstale":"2024-04-28T12:39:12.511320+0000","last_undegraded":"2024-04-28T12:39:12.511320+0000","last_fullsized":"2024-04-28T12:39:12.511320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:39:14.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093214+0000","last_change":"2024-04-28T12:39:10.080059+0000","last_active":"2024-04-28T12:39:12.093214+0000","last_peered":"2024-04-28T12:39:12.093214+0000","last_clean":"2024-04-28T12:39:12.093214+0000","last_became_active":"2024-04-28T12:39:10.079700+0000","last_became_peered":"2024-04-28T12:39:10.079700+0000","last_unstale":"2024-04-28T12:39:12.093214+0000","last_undegraded":"2024-04-28T12:39:12.093214+0000","last_fullsized":"2024-04-28T12:39:12.093214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:31:44.816764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276729+0000","last_change":"2024-04-28T12:39:11.106286+0000","last_active":"2024-04-28T12:39:13.276729+0000","last_peered":"2024-04-28T12:39:13.276729+0000","last_clean":"2024-04-28T12:39:13.276729+0000","last_became_active":"2024-04-28T12:39:11.105875+0000","last_became_peered":"2024-04-28T12:39:11.105875+0000","last_unstale":"2024-04-28T12:39:13.276729+0000","last_undegraded":"2024-04-28T12:39:13.276729+0000","last_fullsized":"2024-04-28T12:39:13.276729+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:18:19.496186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.357475+0000","last_change":"2024-04-28T12:39:10.099531+0000","last_active":"2024-04-28T12:39:12.357475+0000","last_peered":"2024-04-28T12:39:12.357475+0000","last_clean":"2024-04-28T12:39:12.357475+0000","last_became_active":"2024-04-28T12:39:10.099251+0000","last_became_peered":"2024-04-28T12:39:10.099251+0000","last_unstale":"2024-04-28T12:39:12.357475+0000","last_undegraded":"2024-04-28T12:39:12.357475+0000","last_fullsized":"2024-04-28T12:39:12.357475+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:41.434116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.277263+0000","last_change":"2024-04-28T12:39:11.112722+0000","last_active":"2024-04-28T12:39:13.277263+0000","last_peered":"2024-04-28T12:39:13.277263+0000","last_clean":"2024-04-28T12:39:13.277263+0000","last_became_active":"2024-04-28T12:39:11.109848+0000","last_became_peered":"2024-04-28T12:39:11.109848+0000","last_unstale":"2024-04-28T12:39:13.277263+0000","last_undegraded":"2024-04-28T12:39:13.277263+0000","last_fullsized":"2024-04-28T12:39:13.277263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:22.911705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094622+0000","last_change":"2024-04-28T12:39:10.103341+0000","last_active":"2024-04-28T12:39:12.094622+0000","last_peered":"2024-04-28T12:39:12.094622+0000","last_clean":"2024-04-28T12:39:12.094622+0000","last_became_active":"2024-04-28T12:39:10.103061+0000","last_became_peered":"2024-04-28T12:39:10.103061+0000","last_unstale":"2024-04-28T12:39:12.094622+0000","last_undegraded":"2024-04-28T12:39:12.094622+0000","last_fullsized":"2024-04-28T12:39:12.094622+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:38:16.272036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511559+0000","last_change":"2024-04-28T12:39:11.096078+0000","last_active":"2024-04-28T12:39:12.511559+0000","last_peered":"2024-04-28T12:39:12.511559+0000","last_clean":"2024-04-28T12:39:12.511559+0000","last_became_active":"2024-04-28T12:39:11.095775+0000","last_became_peered":"2024-04-28T12:39:11.095775+0000","last_unstale":"2024-04-28T12:39:12.511559+0000","last_undegraded":"2024-04-28T12:39:12.511559+0000","last_fullsized":"2024-04-28T12:39:12.511559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:11:04.785848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105293+0000","last_change":"2024-04-28T12:39:10.082613+0000","last_active":"2024-04-28T12:39:13.105293+0000","last_peered":"2024-04-28T12:39:13.105293+0000","last_clean":"2024-04-28T12:39:13.105293+0000","last_became_active":"2024-04-28T12:39:10.082286+0000","last_became_peered":"2024-04-28T12:39:10.082286+0000","last_unstale":"2024-04-28T12:39:13.105293+0000","last_undegraded":"2024-04-28T12:39:13.105293+0000","last_fullsized":"2024-04-28T12:39:13.105293+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:41:23.038524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105180+0000","last_change":"2024-04-28T12:39:11.101878+0000","last_active":"2024-04-28T12:39:13.105180+0000","last_peered":"2024-04-28T12:39:13.105180+0000","last_clean":"2024-04-28T12:39:13.105180+0000","last_became_active":"2024-04-28T12:39:11.096194+0000","last_became_peered":"2024-04-28T12:39:11.096194+0000","last_unstale":"2024-04-28T12:39:13.105180+0000","last_undegraded":"2024-04-28T12:39:13.105180+0000","last_fullsized":"2024-04-28T12:39:13.105180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:48:38.113586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.104858+0000","last_change":"2024-04-28T12:39:10.088421+0000","last_active":"2024-04-28T12:39:13.104858+0000","last_peered":"2024-04-28T12:39:13.104858+0000","last_clean":"2024-04-28T12:39:13.104858+0000","last_became_active":"2024-04-28T12:39:10.088119+0000","last_became_peered":"2024-04-28T12:39:10.088119+0000","last_unstale":"2024-04-28T12:39:13.104858+0000","last_undegraded":"2024-04-28T12:39:13.104858+0000","last_fullsized":"2024-04-28T12:39:13.104858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:18:45.075219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276263+0000","last_change":"2024-04-28T12:39:11.097561+0000","last_active":"2024-04-28T12:39:13.276263+0000","last_peered":"2024-04-28T12:39:13.276263+0000","last_clean":"2024-04-28T12:39:13.276263+0000","last_became_active":"2024-04-28T12:39:11.097323+0000","last_became_peered":"2024-04-28T12:39:11.097323+0000","last_unstale":"2024-04-28T12:39:13.276263+0000","last_undegraded":"2024-04-28T12:39:13.276263+0000","last_fullsized":"2024-04-28T12:39:13.276263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:38:02.354280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093990+0000","last_change":"2024-04-28T12:39:10.081244+0000","last_active":"2024-04-28T12:39:12.093990+0000","last_peered":"2024-04-28T12:39:12.093990+0000","last_clean":"2024-04-28T12:39:12.093990+0000","last_became_active":"2024-04-28T12:39:10.080943+0000","last_became_peered":"2024-04-28T12:39:10.080943+0000","last_unstale":"2024-04-28T12:39:12.093990+0000","last_undegraded":"2024-04-28T12:39:12.093990+0000","last_fullsized":"2024-04-28T12:39:12.093990+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:48.009392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.096789+0000","last_change":"2024-04-28T12:39:11.095515+0000","last_active":"2024-04-28T12:39:12.096789+0000","last_peered":"2024-04-28T12:39:12.096789+0000","last_clean":"2024-04-28T12:39:12.096789+0000","last_became_active":"2024-04-28T12:39:11.095255+0000","last_became_peered":"2024-04-28T12:39:11.095255+0000","last_unstale":"2024-04-28T12:39:12.096789+0000","last_undegraded":"2024-04-28T12:39:12.096789+0000","last_fullsized":"2024-04-28T12:39:12.096789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:47:42.830968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.095556+0000","last_change":"2024-04-28T12:39:10.084354+0000","last_active":"2024-04-28T12:39:12.095556+0000","last_peered":"2024-04-28T12:39:12.095556+0000","last_clean":"2024-04-28T12:39:12.095556+0000","last_became_active":"2024-04-28T12:39:10.083623+0000","last_became_peered":"2024-04-28T12:39:10.083623+0000","last_unstale":"2024-04-28T12:39:12.095556+0000","last_undegraded":"2024-04-28T12:39:12.095556+0000","last_fullsized":"2024-04-28T12:39:12.095556+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:31:06.775995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.277196+0000","last_change":"2024-04-28T12:39:11.110940+0000","last_active":"2024-04-28T12:39:13.277196+0000","last_peered":"2024-04-28T12:39:13.277196+0000","last_clean":"2024-04-28T12:39:13.277196+0000","last_became_active":"2024-04-28T12:39:11.110533+0000","last_became_peered":"2024-04-28T12:39:11.110533+0000","last_unstale":"2024-04-28T12:39:13.277196+0000","last_undegraded":"2024-04-28T12:39:13.277196+0000","last_fullsized":"2024-04-28T12:39:13.277196+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:48:28.059974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.510450+0000","last_change":"2024-04-28T12:39:10.080505+0000","last_active":"2024-04-28T12:39:12.510450+0000","last_peered":"2024-04-28T12:39:12.510450+0000","last_clean":"2024-04-28T12:39:12.510450+0000","last_became_active":"2024-04-28T12:39:10.080178+0000","last_became_peered":"2024-04-28T12:39:10.080178+0000","last_unstale":"2024-04-28T12:39:12.510450+0000","last_undegraded":"2024-04-28T12:39:12.510450+0000","last_fullsized":"2024-04-28T12:39:12.510450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:07:17.221125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356907+0000","last_change":"2024-04-28T12:39:11.108980+0000","last_active":"2024-04-28T12:39:12.356907+0000","last_peered":"2024-04-28T12:39:12.356907+0000","last_clean":"2024-04-28T12:39:12.356907+0000","last_became_active":"2024-04-28T12:39:11.108692+0000","last_became_peered":"2024-04-28T12:39:11.108692+0000","last_unstale":"2024-04-28T12:39:12.356907+0000","last_undegraded":"2024-04-28T12:39:12.356907+0000","last_fullsized":"2024-04-28T12:39:12.356907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:07:42.536830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276974+0000","last_change":"2024-04-28T12:39:10.103090+0000","last_active":"2024-04-28T12:39:13.276974+0000","last_peered":"2024-04-28T12:39:13.276974+0000","last_clean":"2024-04-28T12:39:13.276974+0000","last_became_active":"2024-04-28T12:39:10.102773+0000","last_became_peered":"2024-04-28T12:39:10.102773+0000","last_unstale":"2024-04-28T12:39:13.276974+0000","last_undegraded":"2024-04-28T12:39:13.276974+0000","last_fullsized":"2024-04-28T12:39:13.276974+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:12.923604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276678+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:39:13.276678+0000","last_peered":"2024-04-28T12:39:13.276678+0000","last_clean":"2024-04-28T12:39:13.276678+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:39:13.276678+0000","last_undegraded":"2024-04-28T12:39:13.276678+0000","last_fullsized":"2024-04-28T12:39:13.276678+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":782086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":442368,"data_stored":190618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":446464,"data_stored":190148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":446464,"data_stored":190640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":191682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":190106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":190126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":780438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:39:20.523 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:39:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.736+0000 7fbf38800640 1 Processor -- start 2024-04-28T12:39:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.736+0000 7fbf38800640 1 -- start start 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.737+0000 7fbf38800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbf30145590 0x7fbf300ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.737+0000 7fbf38800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fbf300b3490 0x7fbf3006f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.737+0000 7fbf38800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf3006f560 0x7fbf300d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.737+0000 7fbf38800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fbf300d0100 con 0x7fbf30145590 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.738+0000 7fbf38800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fbf3006f9d0 con 0x7fbf300b3490 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.738+0000 7fbf38800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbf300cf060 con 0x7fbf3006f560 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.738+0000 7fbf35e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbf30145590 0x7fbf300ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.738+0000 7fbf35e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbf30145590 0x7fbf300ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57440/0 (socket says 172.21.15.131:57440) 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.738+0000 7fbf35e00640 1 -- 172.21.15.131:0/2093068412 learned_addr learned my addr 172.21.15.131:0/2093068412 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.738+0000 7fbf35e00640 1 -- 172.21.15.131:0/2093068412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf3006f560 msgr2=0x7fbf300d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.738+0000 7fbf36800640 1 --2- 172.21.15.131:0/2093068412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf3006f560 0x7fbf300d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.738+0000 7fbf35400640 1 --2- 172.21.15.131:0/2093068412 >> v2:172.21.15.157:3300/0 conn(0x7fbf300b3490 0x7fbf3006f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.738+0000 7fbf35e00640 1 --2- 172.21.15.131:0/2093068412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf3006f560 0x7fbf300d0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.738+0000 7fbf35e00640 1 -- 172.21.15.131:0/2093068412 >> v2:172.21.15.157:3300/0 conn(0x7fbf300b3490 msgr2=0x7fbf3006f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.738+0000 7fbf35e00640 1 --2- 172.21.15.131:0/2093068412 >> v2:172.21.15.157:3300/0 conn(0x7fbf300b3490 0x7fbf3006f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.738+0000 7fbf35e00640 1 -- 172.21.15.131:0/2093068412 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf300cf340 con 0x7fbf30145590 2024-04-28T12:39:20.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.739+0000 7fbf35e00640 1 --2- 172.21.15.131:0/2093068412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf30145590 0x7fbf300ce780 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbf30074050 tx=0x7fbf2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fb8ae46acc7b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:20.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.740+0000 7fbf34a00640 1 -- 172.21.15.131:0/2093068412 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24037070 con 0x7fbf30145590 2024-04-28T12:39:20.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.740+0000 7fbf34a00640 1 -- 172.21.15.131:0/2093068412 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf24032440 con 0x7fbf30145590 2024-04-28T12:39:20.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.740+0000 7fbf34a00640 1 -- 172.21.15.131:0/2093068412 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24032810 con 0x7fbf30145590 2024-04-28T12:39:20.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.740+0000 7fbf38800640 1 -- 172.21.15.131:0/2093068412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf30145590 msgr2=0x7fbf300ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:20.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.740+0000 7fbf38800640 1 --2- 172.21.15.131:0/2093068412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf30145590 0x7fbf300ce780 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbf30074050 tx=0x7fbf2402a1e0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.740+0000 7fbf38800640 1 -- 172.21.15.131:0/2093068412 shutdown_connections 2024-04-28T12:39:20.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.740+0000 7fbf38800640 1 --2- 172.21.15.131:0/2093068412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf3006f560 0x7fbf300d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.740+0000 7fbf38800640 1 --2- 172.21.15.131:0/2093068412 >> v2:172.21.15.157:3300/0 conn(0x7fbf300b3490 0x7fbf3006f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.740+0000 7fbf38800640 1 --2- 172.21.15.131:0/2093068412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf30145590 0x7fbf300ce780 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.740+0000 7fbf38800640 1 -- 172.21.15.131:0/2093068412 >> 172.21.15.131:0/2093068412 conn(0x7fbf300b2900 msgr2=0x7fbf300b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.741+0000 7fbf38800640 1 -- 172.21.15.131:0/2093068412 shutdown_connections 2024-04-28T12:39:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.741+0000 7fbf38800640 1 -- 172.21.15.131:0/2093068412 wait complete. 2024-04-28T12:39:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.742+0000 7fbf38800640 1 Processor -- start 2024-04-28T12:39:20.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.742+0000 7fbf38800640 1 -- start start 2024-04-28T12:39:20.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.743+0000 7fbf38800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3006f560 0x7fbf30141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.743+0000 7fbf38800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf300b3490 0x7fbf3015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.743+0000 7fbf38800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf30145590 0x7fbf3015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.743+0000 7fbf38800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbf300cfbe0 con 0x7fbf3006f560 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.743+0000 7fbf38800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf300cf900 con 0x7fbf300b3490 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.743+0000 7fbf38800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbf30145970 con 0x7fbf30145590 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.744+0000 7fbf35e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3006f560 0x7fbf30141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.744+0000 7fbf35e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3006f560 0x7fbf30141100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57452/0 (socket says 172.21.15.131:57452) 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.744+0000 7fbf35e00640 1 -- 172.21.15.131:0/1055330843 learned_addr learned my addr 172.21.15.131:0/1055330843 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.744+0000 7fbf35400640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf300b3490 0x7fbf3015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.744+0000 7fbf35e00640 1 -- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf30145590 msgr2=0x7fbf3015b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.744+0000 7fbf35e00640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf30145590 0x7fbf3015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.744+0000 7fbf35e00640 1 -- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf300b3490 msgr2=0x7fbf3015d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:20.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.744+0000 7fbf35e00640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf300b3490 0x7fbf3015d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.744+0000 7fbf35e00640 1 -- 172.21.15.131:0/1055330843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf30141930 con 0x7fbf3006f560 2024-04-28T12:39:20.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.744+0000 7fbf35e00640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3006f560 0x7fbf30141100 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbf2402a220 tx=0x7fbf240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:20.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.745+0000 7fbf2a000640 1 -- 172.21.15.131:0/1055330843 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24041070 con 0x7fbf3006f560 2024-04-28T12:39:20.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.745+0000 7fbf2a000640 1 -- 172.21.15.131:0/1055330843 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf2403c870 con 0x7fbf3006f560 2024-04-28T12:39:20.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.745+0000 7fbf2a000640 1 -- 172.21.15.131:0/1055330843 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24037040 con 0x7fbf3006f560 2024-04-28T12:39:20.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.745+0000 7fbf38800640 1 -- 172.21.15.131:0/1055330843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf30141c10 con 0x7fbf3006f560 2024-04-28T12:39:20.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.745+0000 7fbf38800640 1 -- 172.21.15.131:0/1055330843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf301421d0 con 0x7fbf3006f560 2024-04-28T12:39:20.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.746+0000 7fbf38800640 1 -- 172.21.15.131:0/1055330843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf30069350 con 0x7fbf3006f560 2024-04-28T12:39:20.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.747+0000 7fbf2a000640 1 -- 172.21.15.131:0/1055330843 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf24044070 con 0x7fbf3006f560 2024-04-28T12:39:20.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.748+0000 7fbf2a000640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbf1406c8f0 0x7fbf1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:20.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.748+0000 7fbf2a000640 1 -- 172.21.15.131:0/1055330843 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fbf240acac0 con 0x7fbf3006f560 2024-04-28T12:39:20.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.748+0000 7fbf35400640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbf1406c8f0 0x7fbf1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:20.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.750+0000 7fbf35400640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbf1406c8f0 0x7fbf1406edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbf20006fd0 tx=0x7fbf20008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:20.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.753+0000 7fbf2a000640 1 -- 172.21.15.131:0/1055330843 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf2407ade0 con 0x7fbf3006f560 2024-04-28T12:39:20.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.948+0000 7fbf38800640 1 -- 172.21.15.131:0/1055330843 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf3013f180 con 0x7fbf1406c8f0 2024-04-28T12:39:20.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.967+0000 7fbf2a000640 1 -- 172.21.15.131:0/1055330843 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7fbf3013f180 con 0x7fbf1406c8f0 2024-04-28T12:39:20.967 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:20.969 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:39:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.974+0000 7fbf38800640 1 -- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbf1406c8f0 msgr2=0x7fbf1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.974+0000 7fbf38800640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbf1406c8f0 0x7fbf1406edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbf20006fd0 tx=0x7fbf20008040 comp rx=0 tx=0).stop 2024-04-28T12:39:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.974+0000 7fbf38800640 1 -- 172.21.15.131:0/1055330843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3006f560 msgr2=0x7fbf30141100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.974+0000 7fbf38800640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3006f560 0x7fbf30141100 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbf2402a220 tx=0x7fbf240046c0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.974+0000 7fbf38800640 1 -- 172.21.15.131:0/1055330843 shutdown_connections 2024-04-28T12:39:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.974+0000 7fbf38800640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbf1406c8f0 0x7fbf1406edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.974+0000 7fbf38800640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf30145590 0x7fbf3015b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.974+0000 7fbf38800640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf300b3490 0x7fbf3015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.974+0000 7fbf38800640 1 --2- 172.21.15.131:0/1055330843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf3006f560 0x7fbf30141100 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.974+0000 7fbf38800640 1 -- 172.21.15.131:0/1055330843 >> 172.21.15.131:0/1055330843 conn(0x7fbf300b2900 msgr2=0x7fbf300735e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.974+0000 7fbf38800640 1 -- 172.21.15.131:0/1055330843 shutdown_connections 2024-04-28T12:39:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:20.974+0000 7fbf38800640 1 -- 172.21.15.131:0/1055330843 wait complete. 2024-04-28T12:39:20.990 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-04-28T12:39:18.960739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237492,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737228,"statfs":{"total":773094113280,"available":772850921472,"internally_reserved":0,"allocated":4759552,"data_stored":2705844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003028"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511760+0000","last_change":"2024-04-28T12:39:11.103632+0000","last_active":"2024-04-28T12:39:12.511760+0000","last_peered":"2024-04-28T12:39:12.511760+0000","last_clean":"2024-04-28T12:39:12.511760+0000","last_became_active":"2024-04-28T12:39:11.102942+0000","last_became_peered":"2024-04-28T12:39:11.102942+0000","last_unstale":"2024-04-28T12:39:12.511760+0000","last_undegraded":"2024-04-28T12:39:12.511760+0000","last_fullsized":"2024-04-28T12:39:12.511760+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:42:35.965759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511257+0000","last_change":"2024-04-28T12:39:11.093817+0000","last_active":"2024-04-28T12:39:12.511257+0000","last_peered":"2024-04-28T12:39:12.511257+0000","last_clean":"2024-04-28T12:39:12.511257+0000","last_became_active":"2024-04-28T12:39:11.089057+0000","last_became_peered":"2024-04-28T12:39:11.089057+0000","last_unstale":"2024-04-28T12:39:12.511257+0000","last_undegraded":"2024-04-28T12:39:12.511257+0000","last_fullsized":"2024-04-28T12:39:12.511257+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:28:06.491386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105034+0000","last_change":"2024-04-28T12:39:11.099167+0000","last_active":"2024-04-28T12:39:13.105034+0000","last_peered":"2024-04-28T12:39:13.105034+0000","last_clean":"2024-04-28T12:39:13.105034+0000","last_became_active":"2024-04-28T12:39:11.098888+0000","last_became_peered":"2024-04-28T12:39:11.098888+0000","last_unstale":"2024-04-28T12:39:13.105034+0000","last_undegraded":"2024-04-28T12:39:13.105034+0000","last_fullsized":"2024-04-28T12:39:13.105034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:38:02.354280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094346+0000","last_change":"2024-04-28T12:39:11.082989+0000","last_active":"2024-04-28T12:39:12.094346+0000","last_peered":"2024-04-28T12:39:12.094346+0000","last_clean":"2024-04-28T12:39:12.094346+0000","last_became_active":"2024-04-28T12:39:11.082635+0000","last_became_peered":"2024-04-28T12:39:11.082635+0000","last_unstale":"2024-04-28T12:39:12.094346+0000","last_undegraded":"2024-04-28T12:39:12.094346+0000","last_fullsized":"2024-04-28T12:39:12.094346+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:46:57.693015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093710+0000","last_change":"2024-04-28T12:39:11.082537+0000","last_active":"2024-04-28T12:39:12.093710+0000","last_peered":"2024-04-28T12:39:12.093710+0000","last_clean":"2024-04-28T12:39:12.093710+0000","last_became_active":"2024-04-28T12:39:11.082219+0000","last_became_peered":"2024-04-28T12:39:11.082219+0000","last_unstale":"2024-04-28T12:39:12.093710+0000","last_undegraded":"2024-04-28T12:39:12.093710+0000","last_fullsized":"2024-04-28T12:39:12.093710+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:50:52.412029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276957+0000","last_change":"2024-04-28T12:39:11.103589+0000","last_active":"2024-04-28T12:39:13.276957+0000","last_peered":"2024-04-28T12:39:13.276957+0000","last_clean":"2024-04-28T12:39:13.276957+0000","last_became_active":"2024-04-28T12:39:11.102204+0000","last_became_peered":"2024-04-28T12:39:11.102204+0000","last_unstale":"2024-04-28T12:39:13.276957+0000","last_undegraded":"2024-04-28T12:39:13.276957+0000","last_fullsized":"2024-04-28T12:39:13.276957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:30.942037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.355840+0000","last_change":"2024-04-28T12:39:11.106371+0000","last_active":"2024-04-28T12:39:12.355840+0000","last_peered":"2024-04-28T12:39:12.355840+0000","last_clean":"2024-04-28T12:39:12.355840+0000","last_became_active":"2024-04-28T12:39:11.106005+0000","last_became_peered":"2024-04-28T12:39:11.106005+0000","last_unstale":"2024-04-28T12:39:12.355840+0000","last_undegraded":"2024-04-28T12:39:12.355840+0000","last_fullsized":"2024-04-28T12:39:12.355840+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:27:11.732145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.510817+0000","last_change":"2024-04-28T12:39:11.108164+0000","last_active":"2024-04-28T12:39:12.510817+0000","last_peered":"2024-04-28T12:39:12.510817+0000","last_clean":"2024-04-28T12:39:12.510817+0000","last_became_active":"2024-04-28T12:39:11.107836+0000","last_became_peered":"2024-04-28T12:39:11.107836+0000","last_unstale":"2024-04-28T12:39:12.510817+0000","last_undegraded":"2024-04-28T12:39:12.510817+0000","last_fullsized":"2024-04-28T12:39:12.510817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:48:28.059974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105875+0000","last_change":"2024-04-28T12:39:11.105441+0000","last_active":"2024-04-28T12:39:13.105875+0000","last_peered":"2024-04-28T12:39:13.105875+0000","last_clean":"2024-04-28T12:39:13.105875+0000","last_became_active":"2024-04-28T12:39:11.105063+0000","last_became_peered":"2024-04-28T12:39:11.105063+0000","last_unstale":"2024-04-28T12:39:13.105875+0000","last_undegraded":"2024-04-28T12:39:13.105875+0000","last_fullsized":"2024-04-28T12:39:13.105875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:30.942037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.357011+0000","last_change":"2024-04-28T12:39:11.104694+0000","last_active":"2024-04-28T12:39:12.357011+0000","last_peered":"2024-04-28T12:39:12.357011+0000","last_clean":"2024-04-28T12:39:12.357011+0000","last_became_active":"2024-04-28T12:39:11.104406+0000","last_became_peered":"2024-04-28T12:39:11.104406+0000","last_unstale":"2024-04-28T12:39:12.357011+0000","last_undegraded":"2024-04-28T12:39:12.357011+0000","last_fullsized":"2024-04-28T12:39:12.357011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:41:28.559411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356910+0000","last_change":"2024-04-28T12:39:11.101140+0000","last_active":"2024-04-28T12:39:12.356910+0000","last_peered":"2024-04-28T12:39:12.356910+0000","last_clean":"2024-04-28T12:39:12.356910+0000","last_became_active":"2024-04-28T12:39:11.100193+0000","last_became_peered":"2024-04-28T12:39:11.100193+0000","last_unstale":"2024-04-28T12:39:12.356910+0000","last_undegraded":"2024-04-28T12:39:12.356910+0000","last_fullsized":"2024-04-28T12:39:12.356910+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:11:04.785848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105292+0000","last_change":"2024-04-28T12:39:11.090814+0000","last_active":"2024-04-28T12:39:13.105292+0000","last_peered":"2024-04-28T12:39:13.105292+0000","last_clean":"2024-04-28T12:39:13.105292+0000","last_became_active":"2024-04-28T12:39:11.090509+0000","last_became_peered":"2024-04-28T12:39:11.090509+0000","last_unstale":"2024-04-28T12:39:13.105292+0000","last_undegraded":"2024-04-28T12:39:13.105292+0000","last_fullsized":"2024-04-28T12:39:13.105292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:53:12.218233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356081+0000","last_change":"2024-04-28T12:39:11.110526+0000","last_active":"2024-04-28T12:39:12.356081+0000","last_peered":"2024-04-28T12:39:12.356081+0000","last_clean":"2024-04-28T12:39:12.356081+0000","last_became_active":"2024-04-28T12:39:11.110239+0000","last_became_peered":"2024-04-28T12:39:11.110239+0000","last_unstale":"2024-04-28T12:39:12.356081+0000","last_undegraded":"2024-04-28T12:39:12.356081+0000","last_fullsized":"2024-04-28T12:39:12.356081+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:46:13.934855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.096190+0000","last_change":"2024-04-28T12:39:11.090327+0000","last_active":"2024-04-28T12:39:12.096190+0000","last_peered":"2024-04-28T12:39:12.096190+0000","last_clean":"2024-04-28T12:39:12.096190+0000","last_became_active":"2024-04-28T12:39:11.090002+0000","last_became_peered":"2024-04-28T12:39:11.090002+0000","last_unstale":"2024-04-28T12:39:12.096190+0000","last_undegraded":"2024-04-28T12:39:12.096190+0000","last_fullsized":"2024-04-28T12:39:12.096190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:42.448610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356025+0000","last_change":"2024-04-28T12:39:11.100853+0000","last_active":"2024-04-28T12:39:12.356025+0000","last_peered":"2024-04-28T12:39:12.356025+0000","last_clean":"2024-04-28T12:39:12.356025+0000","last_became_active":"2024-04-28T12:39:11.100566+0000","last_became_peered":"2024-04-28T12:39:11.100566+0000","last_unstale":"2024-04-28T12:39:12.356025+0000","last_undegraded":"2024-04-28T12:39:12.356025+0000","last_fullsized":"2024-04-28T12:39:12.356025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:48:28.059974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276344+0000","last_change":"2024-04-28T12:39:11.113771+0000","last_active":"2024-04-28T12:39:13.276344+0000","last_peered":"2024-04-28T12:39:13.276344+0000","last_clean":"2024-04-28T12:39:13.276344+0000","last_became_active":"2024-04-28T12:39:11.112983+0000","last_became_peered":"2024-04-28T12:39:11.112983+0000","last_unstale":"2024-04-28T12:39:13.276344+0000","last_undegraded":"2024-04-28T12:39:13.276344+0000","last_fullsized":"2024-04-28T12:39:13.276344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:53:12.218233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.277645+0000","last_change":"2024-04-28T12:39:11.114601+0000","last_active":"2024-04-28T12:39:13.277645+0000","last_peered":"2024-04-28T12:39:13.277645+0000","last_clean":"2024-04-28T12:39:13.277645+0000","last_became_active":"2024-04-28T12:39:11.114305+0000","last_became_peered":"2024-04-28T12:39:11.114305+0000","last_unstale":"2024-04-28T12:39:13.277645+0000","last_undegraded":"2024-04-28T12:39:13.277645+0000","last_fullsized":"2024-04-28T12:39:13.277645+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:46:13.934855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.095543+0000","last_change":"2024-04-28T12:39:11.082583+0000","last_active":"2024-04-28T12:39:12.095543+0000","last_peered":"2024-04-28T12:39:12.095543+0000","last_clean":"2024-04-28T12:39:12.095543+0000","last_became_active":"2024-04-28T12:39:11.081714+0000","last_became_peered":"2024-04-28T12:39:11.081714+0000","last_unstale":"2024-04-28T12:39:12.095543+0000","last_undegraded":"2024-04-28T12:39:12.095543+0000","last_fullsized":"2024-04-28T12:39:12.095543+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:28:23.770671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094644+0000","last_change":"2024-04-28T12:39:11.095705+0000","last_active":"2024-04-28T12:39:12.094644+0000","last_peered":"2024-04-28T12:39:12.094644+0000","last_clean":"2024-04-28T12:39:12.094644+0000","last_became_active":"2024-04-28T12:39:11.095446+0000","last_became_peered":"2024-04-28T12:39:11.095446+0000","last_unstale":"2024-04-28T12:39:12.094644+0000","last_undegraded":"2024-04-28T12:39:12.094644+0000","last_fullsized":"2024-04-28T12:39:12.094644+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:58:12.246960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093824+0000","last_change":"2024-04-28T12:39:11.080533+0000","last_active":"2024-04-28T12:39:12.093824+0000","last_peered":"2024-04-28T12:39:12.093824+0000","last_clean":"2024-04-28T12:39:12.093824+0000","last_became_active":"2024-04-28T12:39:11.080087+0000","last_became_peered":"2024-04-28T12:39:11.080087+0000","last_unstale":"2024-04-28T12:39:12.093824+0000","last_undegraded":"2024-04-28T12:39:12.093824+0000","last_fullsized":"2024-04-28T12:39:12.093824+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:58.590224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356636+0000","last_change":"2024-04-28T12:39:11.108814+0000","last_active":"2024-04-28T12:39:12.356636+0000","last_peered":"2024-04-28T12:39:12.356636+0000","last_clean":"2024-04-28T12:39:12.356636+0000","last_became_active":"2024-04-28T12:39:11.108471+0000","last_became_peered":"2024-04-28T12:39:11.108471+0000","last_unstale":"2024-04-28T12:39:12.356636+0000","last_undegraded":"2024-04-28T12:39:12.356636+0000","last_fullsized":"2024-04-28T12:39:12.356636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:28:06.491386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.097492+0000","last_change":"2024-04-28T12:39:11.085015+0000","last_active":"2024-04-28T12:39:12.097492+0000","last_peered":"2024-04-28T12:39:12.097492+0000","last_clean":"2024-04-28T12:39:12.097492+0000","last_became_active":"2024-04-28T12:39:11.084662+0000","last_became_peered":"2024-04-28T12:39:11.084662+0000","last_unstale":"2024-04-28T12:39:12.097492+0000","last_undegraded":"2024-04-28T12:39:12.097492+0000","last_fullsized":"2024-04-28T12:39:12.097492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:07:18.235619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276444+0000","last_change":"2024-04-28T12:39:11.113671+0000","last_active":"2024-04-28T12:39:13.276444+0000","last_peered":"2024-04-28T12:39:13.276444+0000","last_clean":"2024-04-28T12:39:13.276444+0000","last_became_active":"2024-04-28T12:39:11.113172+0000","last_became_peered":"2024-04-28T12:39:11.113172+0000","last_unstale":"2024-04-28T12:39:13.276444+0000","last_undegraded":"2024-04-28T12:39:13.276444+0000","last_fullsized":"2024-04-28T12:39:13.276444+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:18:19.496186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093764+0000","last_change":"2024-04-28T12:39:11.107323+0000","last_active":"2024-04-28T12:39:12.093764+0000","last_peered":"2024-04-28T12:39:12.093764+0000","last_clean":"2024-04-28T12:39:12.093764+0000","last_became_active":"2024-04-28T12:39:11.107046+0000","last_became_peered":"2024-04-28T12:39:11.107046+0000","last_unstale":"2024-04-28T12:39:12.093764+0000","last_undegraded":"2024-04-28T12:39:12.093764+0000","last_fullsized":"2024-04-28T12:39:12.093764+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:09:14.004417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.095506+0000","last_change":"2024-04-28T12:39:11.090831+0000","last_active":"2024-04-28T12:39:12.095506+0000","last_peered":"2024-04-28T12:39:12.095506+0000","last_clean":"2024-04-28T12:39:12.095506+0000","last_became_active":"2024-04-28T12:39:11.090536+0000","last_became_peered":"2024-04-28T12:39:11.090536+0000","last_unstale":"2024-04-28T12:39:12.095506+0000","last_undegraded":"2024-04-28T12:39:12.095506+0000","last_fullsized":"2024-04-28T12:39:12.095506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:47:42.830968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.277132+0000","last_change":"2024-04-28T12:39:11.108464+0000","last_active":"2024-04-28T12:39:13.277132+0000","last_peered":"2024-04-28T12:39:13.277132+0000","last_clean":"2024-04-28T12:39:13.277132+0000","last_became_active":"2024-04-28T12:39:11.108192+0000","last_became_peered":"2024-04-28T12:39:11.108192+0000","last_unstale":"2024-04-28T12:39:13.277132+0000","last_undegraded":"2024-04-28T12:39:13.277132+0000","last_fullsized":"2024-04-28T12:39:13.277132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:19:24.083380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094967+0000","last_change":"2024-04-28T12:39:11.083774+0000","last_active":"2024-04-28T12:39:12.094967+0000","last_peered":"2024-04-28T12:39:12.094967+0000","last_clean":"2024-04-28T12:39:12.094967+0000","last_became_active":"2024-04-28T12:39:11.083507+0000","last_became_peered":"2024-04-28T12:39:11.083507+0000","last_unstale":"2024-04-28T12:39:12.094967+0000","last_undegraded":"2024-04-28T12:39:12.094967+0000","last_fullsized":"2024-04-28T12:39:12.094967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:42.448610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356798+0000","last_change":"2024-04-28T12:39:11.106902+0000","last_active":"2024-04-28T12:39:12.356798+0000","last_peered":"2024-04-28T12:39:12.356798+0000","last_clean":"2024-04-28T12:39:12.356798+0000","last_became_active":"2024-04-28T12:39:11.106618+0000","last_became_peered":"2024-04-28T12:39:11.106618+0000","last_unstale":"2024-04-28T12:39:12.356798+0000","last_undegraded":"2024-04-28T12:39:12.356798+0000","last_fullsized":"2024-04-28T12:39:12.356798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:39:14.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093895+0000","last_change":"2024-04-28T12:39:11.092014+0000","last_active":"2024-04-28T12:39:12.093895+0000","last_peered":"2024-04-28T12:39:12.093895+0000","last_clean":"2024-04-28T12:39:12.093895+0000","last_became_active":"2024-04-28T12:39:11.091756+0000","last_became_peered":"2024-04-28T12:39:11.091756+0000","last_unstale":"2024-04-28T12:39:12.093895+0000","last_undegraded":"2024-04-28T12:39:12.093895+0000","last_fullsized":"2024-04-28T12:39:12.093895+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:17.650325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094109+0000","last_change":"2024-04-28T12:39:11.091584+0000","last_active":"2024-04-28T12:39:12.094109+0000","last_peered":"2024-04-28T12:39:12.094109+0000","last_clean":"2024-04-28T12:39:12.094109+0000","last_became_active":"2024-04-28T12:39:11.091313+0000","last_became_peered":"2024-04-28T12:39:11.091313+0000","last_unstale":"2024-04-28T12:39:12.094109+0000","last_undegraded":"2024-04-28T12:39:12.094109+0000","last_fullsized":"2024-04-28T12:39:12.094109+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:13.938098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276580+0000","last_change":"2024-04-28T12:39:11.114417+0000","last_active":"2024-04-28T12:39:13.276580+0000","last_peered":"2024-04-28T12:39:13.276580+0000","last_clean":"2024-04-28T12:39:13.276580+0000","last_became_active":"2024-04-28T12:39:11.113295+0000","last_became_peered":"2024-04-28T12:39:11.113295+0000","last_unstale":"2024-04-28T12:39:13.276580+0000","last_undegraded":"2024-04-28T12:39:13.276580+0000","last_fullsized":"2024-04-28T12:39:13.276580+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:19:24.083380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276447+0000","last_change":"2024-04-28T12:39:11.088429+0000","last_active":"2024-04-28T12:39:13.276447+0000","last_peered":"2024-04-28T12:39:13.276447+0000","last_clean":"2024-04-28T12:39:13.276447+0000","last_became_active":"2024-04-28T12:39:11.088135+0000","last_became_peered":"2024-04-28T12:39:11.088135+0000","last_unstale":"2024-04-28T12:39:13.276447+0000","last_undegraded":"2024-04-28T12:39:13.276447+0000","last_fullsized":"2024-04-28T12:39:13.276447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:48:38.113586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276988+0000","last_change":"2024-04-28T12:39:11.115910+0000","last_active":"2024-04-28T12:39:13.276988+0000","last_peered":"2024-04-28T12:39:13.276988+0000","last_clean":"2024-04-28T12:39:13.276988+0000","last_became_active":"2024-04-28T12:39:11.115648+0000","last_became_peered":"2024-04-28T12:39:11.115648+0000","last_unstale":"2024-04-28T12:39:13.276988+0000","last_undegraded":"2024-04-28T12:39:13.276988+0000","last_fullsized":"2024-04-28T12:39:13.276988+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:27:11.732145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.095650+0000","last_change":"2024-04-28T12:39:10.095609+0000","last_active":"2024-04-28T12:39:12.095650+0000","last_peered":"2024-04-28T12:39:12.095650+0000","last_clean":"2024-04-28T12:39:12.095650+0000","last_became_active":"2024-04-28T12:39:10.092425+0000","last_became_peered":"2024-04-28T12:39:10.092425+0000","last_unstale":"2024-04-28T12:39:12.095650+0000","last_undegraded":"2024-04-28T12:39:12.095650+0000","last_fullsized":"2024-04-28T12:39:12.095650+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:59:52.609458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276621+0000","last_change":"2024-04-28T12:39:11.109368+0000","last_active":"2024-04-28T12:39:13.276621+0000","last_peered":"2024-04-28T12:39:13.276621+0000","last_clean":"2024-04-28T12:39:13.276621+0000","last_became_active":"2024-04-28T12:39:11.109010+0000","last_became_peered":"2024-04-28T12:39:11.109010+0000","last_unstale":"2024-04-28T12:39:13.276621+0000","last_undegraded":"2024-04-28T12:39:13.276621+0000","last_fullsized":"2024-04-28T12:39:13.276621+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:53:12.218233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105370+0000","last_change":"2024-04-28T12:39:10.091734+0000","last_active":"2024-04-28T12:39:13.105370+0000","last_peered":"2024-04-28T12:39:13.105370+0000","last_clean":"2024-04-28T12:39:13.105370+0000","last_became_active":"2024-04-28T12:39:10.091472+0000","last_became_peered":"2024-04-28T12:39:10.091472+0000","last_unstale":"2024-04-28T12:39:13.105370+0000","last_undegraded":"2024-04-28T12:39:13.105370+0000","last_fullsized":"2024-04-28T12:39:13.105370+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:31:06.775995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.097917+0000","last_change":"2024-04-28T12:39:11.088057+0000","last_active":"2024-04-28T12:39:12.097917+0000","last_peered":"2024-04-28T12:39:12.097917+0000","last_clean":"2024-04-28T12:39:12.097917+0000","last_became_active":"2024-04-28T12:39:11.087786+0000","last_became_peered":"2024-04-28T12:39:11.087786+0000","last_unstale":"2024-04-28T12:39:12.097917+0000","last_undegraded":"2024-04-28T12:39:12.097917+0000","last_fullsized":"2024-04-28T12:39:12.097917+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:37:11.157161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276852+0000","last_change":"2024-04-28T12:39:10.086242+0000","last_active":"2024-04-28T12:39:13.276852+0000","last_peered":"2024-04-28T12:39:13.276852+0000","last_clean":"2024-04-28T12:39:13.276852+0000","last_became_active":"2024-04-28T12:39:10.085476+0000","last_became_peered":"2024-04-28T12:39:10.085476+0000","last_unstale":"2024-04-28T12:39:13.276852+0000","last_undegraded":"2024-04-28T12:39:13.276852+0000","last_fullsized":"2024-04-28T12:39:13.276852+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:41.434116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511762+0000","last_change":"2024-04-28T12:39:11.104584+0000","last_active":"2024-04-28T12:39:12.511762+0000","last_peered":"2024-04-28T12:39:12.511762+0000","last_clean":"2024-04-28T12:39:12.511762+0000","last_became_active":"2024-04-28T12:39:11.103351+0000","last_became_peered":"2024-04-28T12:39:11.103351+0000","last_unstale":"2024-04-28T12:39:12.511762+0000","last_undegraded":"2024-04-28T12:39:12.511762+0000","last_fullsized":"2024-04-28T12:39:12.511762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:41:28.559411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.097159+0000","last_change":"2024-04-28T12:39:10.088525+0000","last_active":"2024-04-28T12:39:12.097159+0000","last_peered":"2024-04-28T12:39:12.097159+0000","last_clean":"2024-04-28T12:39:12.097159+0000","last_became_active":"2024-04-28T12:39:10.088154+0000","last_became_peered":"2024-04-28T12:39:10.088154+0000","last_unstale":"2024-04-28T12:39:12.097159+0000","last_undegraded":"2024-04-28T12:39:12.097159+0000","last_fullsized":"2024-04-28T12:39:12.097159+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:32:09.645564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.104651+0000","last_change":"2024-04-28T12:39:11.094511+0000","last_active":"2024-04-28T12:39:13.104651+0000","last_peered":"2024-04-28T12:39:13.104651+0000","last_clean":"2024-04-28T12:39:13.104651+0000","last_became_active":"2024-04-28T12:39:11.094246+0000","last_became_peered":"2024-04-28T12:39:11.094246+0000","last_unstale":"2024-04-28T12:39:13.104651+0000","last_undegraded":"2024-04-28T12:39:13.104651+0000","last_fullsized":"2024-04-28T12:39:13.104651+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:22:48.363451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356260+0000","last_change":"2024-04-28T12:39:10.096517+0000","last_active":"2024-04-28T12:39:12.356260+0000","last_peered":"2024-04-28T12:39:12.356260+0000","last_clean":"2024-04-28T12:39:12.356260+0000","last_became_active":"2024-04-28T12:39:10.096237+0000","last_became_peered":"2024-04-28T12:39:10.096237+0000","last_unstale":"2024-04-28T12:39:12.356260+0000","last_undegraded":"2024-04-28T12:39:12.356260+0000","last_fullsized":"2024-04-28T12:39:12.356260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:16.635831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356315+0000","last_change":"2024-04-28T12:39:11.110905+0000","last_active":"2024-04-28T12:39:12.356315+0000","last_peered":"2024-04-28T12:39:12.356315+0000","last_clean":"2024-04-28T12:39:12.356315+0000","last_became_active":"2024-04-28T12:39:11.110618+0000","last_became_peered":"2024-04-28T12:39:11.110618+0000","last_unstale":"2024-04-28T12:39:12.356315+0000","last_undegraded":"2024-04-28T12:39:12.356315+0000","last_fullsized":"2024-04-28T12:39:12.356315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:07.039610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276058+0000","last_change":"2024-04-28T12:39:10.100906+0000","last_active":"2024-04-28T12:39:13.276058+0000","last_peered":"2024-04-28T12:39:13.276058+0000","last_clean":"2024-04-28T12:39:13.276058+0000","last_became_active":"2024-04-28T12:39:10.100163+0000","last_became_peered":"2024-04-28T12:39:10.100163+0000","last_unstale":"2024-04-28T12:39:13.276058+0000","last_undegraded":"2024-04-28T12:39:13.276058+0000","last_fullsized":"2024-04-28T12:39:13.276058+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:16.635831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.096809+0000","last_change":"2024-04-28T12:39:11.099558+0000","last_active":"2024-04-28T12:39:12.096809+0000","last_peered":"2024-04-28T12:39:12.096809+0000","last_clean":"2024-04-28T12:39:12.096809+0000","last_became_active":"2024-04-28T12:39:11.097507+0000","last_became_peered":"2024-04-28T12:39:11.097507+0000","last_unstale":"2024-04-28T12:39:12.096809+0000","last_undegraded":"2024-04-28T12:39:12.096809+0000","last_fullsized":"2024-04-28T12:39:12.096809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:28:23.770671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276045+0000","last_change":"2024-04-28T12:39:10.102757+0000","last_active":"2024-04-28T12:39:13.276045+0000","last_peered":"2024-04-28T12:39:13.276045+0000","last_clean":"2024-04-28T12:39:13.276045+0000","last_became_active":"2024-04-28T12:39:10.102457+0000","last_became_peered":"2024-04-28T12:39:10.102457+0000","last_unstale":"2024-04-28T12:39:13.276045+0000","last_undegraded":"2024-04-28T12:39:13.276045+0000","last_fullsized":"2024-04-28T12:39:13.276045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:16.635831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.510901+0000","last_change":"2024-04-28T12:39:11.107070+0000","last_active":"2024-04-28T12:39:12.510901+0000","last_peered":"2024-04-28T12:39:12.510901+0000","last_clean":"2024-04-28T12:39:12.510901+0000","last_became_active":"2024-04-28T12:39:11.106734+0000","last_became_peered":"2024-04-28T12:39:11.106734+0000","last_unstale":"2024-04-28T12:39:12.510901+0000","last_undegraded":"2024-04-28T12:39:12.510901+0000","last_fullsized":"2024-04-28T12:39:12.510901+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:46:13.934855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094998+0000","last_change":"2024-04-28T12:39:10.091842+0000","last_active":"2024-04-28T12:39:12.094998+0000","last_peered":"2024-04-28T12:39:12.094998+0000","last_clean":"2024-04-28T12:39:12.094998+0000","last_became_active":"2024-04-28T12:39:10.091502+0000","last_became_peered":"2024-04-28T12:39:10.091502+0000","last_unstale":"2024-04-28T12:39:12.094998+0000","last_undegraded":"2024-04-28T12:39:12.094998+0000","last_fullsized":"2024-04-28T12:39:12.094998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:48.009392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094560+0000","last_change":"2024-04-28T12:39:11.082913+0000","last_active":"2024-04-28T12:39:12.094560+0000","last_peered":"2024-04-28T12:39:12.094560+0000","last_clean":"2024-04-28T12:39:12.094560+0000","last_became_active":"2024-04-28T12:39:11.081913+0000","last_became_peered":"2024-04-28T12:39:11.081913+0000","last_unstale":"2024-04-28T12:39:12.094560+0000","last_undegraded":"2024-04-28T12:39:12.094560+0000","last_fullsized":"2024-04-28T12:39:12.094560+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:57:36.240279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511835+0000","last_change":"2024-04-28T12:39:10.080378+0000","last_active":"2024-04-28T12:39:12.511835+0000","last_peered":"2024-04-28T12:39:12.511835+0000","last_clean":"2024-04-28T12:39:12.511835+0000","last_became_active":"2024-04-28T12:39:10.080103+0000","last_became_peered":"2024-04-28T12:39:10.080103+0000","last_unstale":"2024-04-28T12:39:12.511835+0000","last_undegraded":"2024-04-28T12:39:12.511835+0000","last_fullsized":"2024-04-28T12:39:12.511835+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:57:30.849560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276902+0000","last_change":"2024-04-28T12:39:11.109905+0000","last_active":"2024-04-28T12:39:13.276902+0000","last_peered":"2024-04-28T12:39:13.276902+0000","last_clean":"2024-04-28T12:39:13.276902+0000","last_became_active":"2024-04-28T12:39:11.109423+0000","last_became_peered":"2024-04-28T12:39:11.109423+0000","last_unstale":"2024-04-28T12:39:13.276902+0000","last_undegraded":"2024-04-28T12:39:13.276902+0000","last_fullsized":"2024-04-28T12:39:13.276902+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:37:34.902184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.357268+0000","last_change":"2024-04-28T12:39:10.095840+0000","last_active":"2024-04-28T12:39:12.357268+0000","last_peered":"2024-04-28T12:39:12.357268+0000","last_clean":"2024-04-28T12:39:12.357268+0000","last_became_active":"2024-04-28T12:39:10.095551+0000","last_became_peered":"2024-04-28T12:39:10.095551+0000","last_unstale":"2024-04-28T12:39:12.357268+0000","last_undegraded":"2024-04-28T12:39:12.357268+0000","last_fullsized":"2024-04-28T12:39:12.357268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:47:41.816474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105575+0000","last_change":"2024-04-28T12:39:11.102650+0000","last_active":"2024-04-28T12:39:13.105575+0000","last_peered":"2024-04-28T12:39:13.105575+0000","last_clean":"2024-04-28T12:39:13.105575+0000","last_became_active":"2024-04-28T12:39:11.102392+0000","last_became_peered":"2024-04-28T12:39:11.102392+0000","last_unstale":"2024-04-28T12:39:13.105575+0000","last_undegraded":"2024-04-28T12:39:13.105575+0000","last_fullsized":"2024-04-28T12:39:13.105575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:18:19.496186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.357068+0000","last_change":"2024-04-28T12:39:10.098882+0000","last_active":"2024-04-28T12:39:12.357068+0000","last_peered":"2024-04-28T12:39:12.357068+0000","last_clean":"2024-04-28T12:39:12.357068+0000","last_became_active":"2024-04-28T12:39:10.098533+0000","last_became_peered":"2024-04-28T12:39:10.098533+0000","last_unstale":"2024-04-28T12:39:12.357068+0000","last_undegraded":"2024-04-28T12:39:12.357068+0000","last_fullsized":"2024-04-28T12:39:12.357068+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:28:22.756177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.097931+0000","last_change":"2024-04-28T12:39:11.086995+0000","last_active":"2024-04-28T12:39:12.097931+0000","last_peered":"2024-04-28T12:39:12.097931+0000","last_clean":"2024-04-28T12:39:12.097931+0000","last_became_active":"2024-04-28T12:39:11.086470+0000","last_became_peered":"2024-04-28T12:39:11.086470+0000","last_unstale":"2024-04-28T12:39:12.097931+0000","last_undegraded":"2024-04-28T12:39:12.097931+0000","last_fullsized":"2024-04-28T12:39:12.097931+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:22:48.363451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511090+0000","last_change":"2024-04-28T12:39:10.088368+0000","last_active":"2024-04-28T12:39:12.511090+0000","last_peered":"2024-04-28T12:39:12.511090+0000","last_clean":"2024-04-28T12:39:12.511090+0000","last_became_active":"2024-04-28T12:39:10.087872+0000","last_became_peered":"2024-04-28T12:39:10.087872+0000","last_unstale":"2024-04-28T12:39:12.511090+0000","last_undegraded":"2024-04-28T12:39:12.511090+0000","last_fullsized":"2024-04-28T12:39:12.511090+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:28:22.756177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.104697+0000","last_change":"2024-04-28T12:39:11.103093+0000","last_active":"2024-04-28T12:39:13.104697+0000","last_peered":"2024-04-28T12:39:13.104697+0000","last_clean":"2024-04-28T12:39:13.104697+0000","last_became_active":"2024-04-28T12:39:11.102769+0000","last_became_peered":"2024-04-28T12:39:11.102769+0000","last_unstale":"2024-04-28T12:39:13.104697+0000","last_undegraded":"2024-04-28T12:39:13.104697+0000","last_fullsized":"2024-04-28T12:39:13.104697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:56:39.106313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511218+0000","last_change":"2024-04-28T12:39:10.078842+0000","last_active":"2024-04-28T12:39:12.511218+0000","last_peered":"2024-04-28T12:39:12.511218+0000","last_clean":"2024-04-28T12:39:12.511218+0000","last_became_active":"2024-04-28T12:39:10.078571+0000","last_became_peered":"2024-04-28T12:39:10.078571+0000","last_unstale":"2024-04-28T12:39:12.511218+0000","last_undegraded":"2024-04-28T12:39:12.511218+0000","last_fullsized":"2024-04-28T12:39:12.511218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:37:10.142667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.510809+0000","last_change":"2024-04-28T12:39:11.096983+0000","last_active":"2024-04-28T12:39:12.510809+0000","last_peered":"2024-04-28T12:39:12.510809+0000","last_clean":"2024-04-28T12:39:12.510809+0000","last_became_active":"2024-04-28T12:39:11.096674+0000","last_became_peered":"2024-04-28T12:39:11.096674+0000","last_unstale":"2024-04-28T12:39:12.510809+0000","last_undegraded":"2024-04-28T12:39:12.510809+0000","last_fullsized":"2024-04-28T12:39:12.510809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:22.911705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276402+0000","last_change":"2024-04-28T12:39:10.092734+0000","last_active":"2024-04-28T12:39:13.276402+0000","last_peered":"2024-04-28T12:39:13.276402+0000","last_clean":"2024-04-28T12:39:13.276402+0000","last_became_active":"2024-04-28T12:39:10.092410+0000","last_became_peered":"2024-04-28T12:39:10.092410+0000","last_unstale":"2024-04-28T12:39:13.276402+0000","last_undegraded":"2024-04-28T12:39:13.276402+0000","last_fullsized":"2024-04-28T12:39:13.276402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:12.923604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276436+0000","last_change":"2024-04-28T12:39:11.112853+0000","last_active":"2024-04-28T12:39:13.276436+0000","last_peered":"2024-04-28T12:39:13.276436+0000","last_clean":"2024-04-28T12:39:13.276436+0000","last_became_active":"2024-04-28T12:39:11.112549+0000","last_became_peered":"2024-04-28T12:39:11.112549+0000","last_unstale":"2024-04-28T12:39:13.276436+0000","last_undegraded":"2024-04-28T12:39:13.276436+0000","last_fullsized":"2024-04-28T12:39:13.276436+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:30.942037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094048+0000","last_change":"2024-04-28T12:39:10.089193+0000","last_active":"2024-04-28T12:39:12.094048+0000","last_peered":"2024-04-28T12:39:12.094048+0000","last_clean":"2024-04-28T12:39:12.094048+0000","last_became_active":"2024-04-28T12:39:10.088837+0000","last_became_peered":"2024-04-28T12:39:10.088837+0000","last_unstale":"2024-04-28T12:39:12.094048+0000","last_undegraded":"2024-04-28T12:39:12.094048+0000","last_fullsized":"2024-04-28T12:39:12.094048+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:58:11.232466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511072+0000","last_change":"2024-04-28T12:39:11.098372+0000","last_active":"2024-04-28T12:39:12.511072+0000","last_peered":"2024-04-28T12:39:12.511072+0000","last_clean":"2024-04-28T12:39:12.511072+0000","last_became_active":"2024-04-28T12:39:11.098095+0000","last_became_peered":"2024-04-28T12:39:11.098095+0000","last_unstale":"2024-04-28T12:39:12.511072+0000","last_undegraded":"2024-04-28T12:39:12.511072+0000","last_fullsized":"2024-04-28T12:39:12.511072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:19:07.039610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276240+0000","last_change":"2024-04-28T12:39:10.100390+0000","last_active":"2024-04-28T12:39:13.276240+0000","last_peered":"2024-04-28T12:39:13.276240+0000","last_clean":"2024-04-28T12:39:13.276240+0000","last_became_active":"2024-04-28T12:39:10.100008+0000","last_became_peered":"2024-04-28T12:39:10.100008+0000","last_unstale":"2024-04-28T12:39:13.276240+0000","last_undegraded":"2024-04-28T12:39:13.276240+0000","last_fullsized":"2024-04-28T12:39:13.276240+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:57.575730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511653+0000","last_change":"2024-04-28T12:39:11.099956+0000","last_active":"2024-04-28T12:39:12.511653+0000","last_peered":"2024-04-28T12:39:12.511653+0000","last_clean":"2024-04-28T12:39:12.511653+0000","last_became_active":"2024-04-28T12:39:11.099709+0000","last_became_peered":"2024-04-28T12:39:11.099709+0000","last_unstale":"2024-04-28T12:39:12.511653+0000","last_undegraded":"2024-04-28T12:39:12.511653+0000","last_fullsized":"2024-04-28T12:39:12.511653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:07:42.536830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356803+0000","last_change":"2024-04-28T12:39:10.094644+0000","last_active":"2024-04-28T12:39:12.356803+0000","last_peered":"2024-04-28T12:39:12.356803+0000","last_clean":"2024-04-28T12:39:12.356803+0000","last_became_active":"2024-04-28T12:39:10.093807+0000","last_became_peered":"2024-04-28T12:39:10.093807+0000","last_unstale":"2024-04-28T12:39:12.356803+0000","last_undegraded":"2024-04-28T12:39:12.356803+0000","last_fullsized":"2024-04-28T12:39:12.356803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:12.923604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.357070+0000","last_change":"2024-04-28T12:39:11.112724+0000","last_active":"2024-04-28T12:39:12.357070+0000","last_peered":"2024-04-28T12:39:12.357070+0000","last_clean":"2024-04-28T12:39:12.357070+0000","last_became_active":"2024-04-28T12:39:11.112245+0000","last_became_peered":"2024-04-28T12:39:11.112245+0000","last_unstale":"2024-04-28T12:39:12.357070+0000","last_undegraded":"2024-04-28T12:39:12.357070+0000","last_fullsized":"2024-04-28T12:39:12.357070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:42:35.965759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.097939+0000","last_change":"2024-04-28T12:39:10.098183+0000","last_active":"2024-04-28T12:39:12.097939+0000","last_peered":"2024-04-28T12:39:12.097939+0000","last_clean":"2024-04-28T12:39:12.097939+0000","last_became_active":"2024-04-28T12:39:10.097444+0000","last_became_peered":"2024-04-28T12:39:10.097444+0000","last_unstale":"2024-04-28T12:39:12.097939+0000","last_undegraded":"2024-04-28T12:39:12.097939+0000","last_fullsized":"2024-04-28T12:39:12.097939+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:59:52.609458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.097788+0000","last_change":"2024-04-28T12:39:11.103256+0000","last_active":"2024-04-28T12:39:12.097788+0000","last_peered":"2024-04-28T12:39:12.097788+0000","last_clean":"2024-04-28T12:39:12.097788+0000","last_became_active":"2024-04-28T12:39:11.102960+0000","last_became_peered":"2024-04-28T12:39:11.102960+0000","last_unstale":"2024-04-28T12:39:12.097788+0000","last_undegraded":"2024-04-28T12:39:12.097788+0000","last_fullsized":"2024-04-28T12:39:12.097788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:57:31.864054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094781+0000","last_change":"2024-04-28T12:39:10.097191+0000","last_active":"2024-04-28T12:39:12.094781+0000","last_peered":"2024-04-28T12:39:12.094781+0000","last_clean":"2024-04-28T12:39:12.094781+0000","last_became_active":"2024-04-28T12:39:10.096896+0000","last_became_peered":"2024-04-28T12:39:10.096896+0000","last_unstale":"2024-04-28T12:39:12.094781+0000","last_undegraded":"2024-04-28T12:39:12.094781+0000","last_fullsized":"2024-04-28T12:39:12.094781+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:54:39.157609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094576+0000","last_change":"2024-04-28T12:39:11.099144+0000","last_active":"2024-04-28T12:39:12.094576+0000","last_peered":"2024-04-28T12:39:12.094576+0000","last_clean":"2024-04-28T12:39:12.094576+0000","last_became_active":"2024-04-28T12:39:11.098133+0000","last_became_peered":"2024-04-28T12:39:11.098133+0000","last_unstale":"2024-04-28T12:39:12.094576+0000","last_undegraded":"2024-04-28T12:39:12.094576+0000","last_fullsized":"2024-04-28T12:39:12.094576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:58.053771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276458+0000","last_change":"2024-04-28T12:39:10.085794+0000","last_active":"2024-04-28T12:39:13.276458+0000","last_peered":"2024-04-28T12:39:13.276458+0000","last_clean":"2024-04-28T12:39:13.276458+0000","last_became_active":"2024-04-28T12:39:10.085462+0000","last_became_peered":"2024-04-28T12:39:10.085462+0000","last_unstale":"2024-04-28T12:39:13.276458+0000","last_undegraded":"2024-04-28T12:39:13.276458+0000","last_fullsized":"2024-04-28T12:39:13.276458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:53:57.575730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276847+0000","last_change":"2024-04-28T12:39:11.113293+0000","last_active":"2024-04-28T12:39:13.276847+0000","last_peered":"2024-04-28T12:39:13.276847+0000","last_clean":"2024-04-28T12:39:13.276847+0000","last_became_active":"2024-04-28T12:39:11.112505+0000","last_became_peered":"2024-04-28T12:39:11.112505+0000","last_unstale":"2024-04-28T12:39:13.276847+0000","last_undegraded":"2024-04-28T12:39:13.276847+0000","last_fullsized":"2024-04-28T12:39:13.276847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:30:01.863945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093761+0000","last_change":"2024-04-28T12:39:10.080044+0000","last_active":"2024-04-28T12:39:12.093761+0000","last_peered":"2024-04-28T12:39:12.093761+0000","last_clean":"2024-04-28T12:39:12.093761+0000","last_became_active":"2024-04-28T12:39:10.079684+0000","last_became_peered":"2024-04-28T12:39:10.079684+0000","last_unstale":"2024-04-28T12:39:12.093761+0000","last_undegraded":"2024-04-28T12:39:12.093761+0000","last_fullsized":"2024-04-28T12:39:12.093761+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:47:56.416266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.104634+0000","last_change":"2024-04-28T12:39:11.106347+0000","last_active":"2024-04-28T12:39:13.104634+0000","last_peered":"2024-04-28T12:39:13.104634+0000","last_clean":"2024-04-28T12:39:13.104634+0000","last_became_active":"2024-04-28T12:39:11.106062+0000","last_became_peered":"2024-04-28T12:39:11.106062+0000","last_unstale":"2024-04-28T12:39:13.104634+0000","last_undegraded":"2024-04-28T12:39:13.104634+0000","last_fullsized":"2024-04-28T12:39:13.104634+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:37:11.157161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093886+0000","last_change":"2024-04-28T12:39:10.084948+0000","last_active":"2024-04-28T12:39:12.093886+0000","last_peered":"2024-04-28T12:39:12.093886+0000","last_clean":"2024-04-28T12:39:12.093886+0000","last_became_active":"2024-04-28T12:39:10.084658+0000","last_became_peered":"2024-04-28T12:39:10.084658+0000","last_unstale":"2024-04-28T12:39:12.093886+0000","last_undegraded":"2024-04-28T12:39:12.093886+0000","last_fullsized":"2024-04-28T12:39:12.093886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:48:58.062822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356067+0000","last_change":"2024-04-28T12:39:11.100927+0000","last_active":"2024-04-28T12:39:12.356067+0000","last_peered":"2024-04-28T12:39:12.356067+0000","last_clean":"2024-04-28T12:39:12.356067+0000","last_became_active":"2024-04-28T12:39:11.097873+0000","last_became_peered":"2024-04-28T12:39:11.097873+0000","last_unstale":"2024-04-28T12:39:12.356067+0000","last_undegraded":"2024-04-28T12:39:12.356067+0000","last_fullsized":"2024-04-28T12:39:12.356067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:22.911705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276599+0000","last_change":"2024-04-28T12:39:10.100038+0000","last_active":"2024-04-28T12:39:13.276599+0000","last_peered":"2024-04-28T12:39:13.276599+0000","last_clean":"2024-04-28T12:39:13.276599+0000","last_became_active":"2024-04-28T12:39:10.098896+0000","last_became_peered":"2024-04-28T12:39:10.098896+0000","last_unstale":"2024-04-28T12:39:13.276599+0000","last_undegraded":"2024-04-28T12:39:13.276599+0000","last_fullsized":"2024-04-28T12:39:13.276599+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:06:28.119829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511320+0000","last_change":"2024-04-28T12:39:11.105099+0000","last_active":"2024-04-28T12:39:12.511320+0000","last_peered":"2024-04-28T12:39:12.511320+0000","last_clean":"2024-04-28T12:39:12.511320+0000","last_became_active":"2024-04-28T12:39:11.104777+0000","last_became_peered":"2024-04-28T12:39:11.104777+0000","last_unstale":"2024-04-28T12:39:12.511320+0000","last_undegraded":"2024-04-28T12:39:12.511320+0000","last_fullsized":"2024-04-28T12:39:12.511320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:39:14.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093214+0000","last_change":"2024-04-28T12:39:10.080059+0000","last_active":"2024-04-28T12:39:12.093214+0000","last_peered":"2024-04-28T12:39:12.093214+0000","last_clean":"2024-04-28T12:39:12.093214+0000","last_became_active":"2024-04-28T12:39:10.079700+0000","last_became_peered":"2024-04-28T12:39:10.079700+0000","last_unstale":"2024-04-28T12:39:12.093214+0000","last_undegraded":"2024-04-28T12:39:12.093214+0000","last_fullsized":"2024-04-28T12:39:12.093214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:31:44.816764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276729+0000","last_change":"2024-04-28T12:39:11.106286+0000","last_active":"2024-04-28T12:39:13.276729+0000","last_peered":"2024-04-28T12:39:13.276729+0000","last_clean":"2024-04-28T12:39:13.276729+0000","last_became_active":"2024-04-28T12:39:11.105875+0000","last_became_peered":"2024-04-28T12:39:11.105875+0000","last_unstale":"2024-04-28T12:39:13.276729+0000","last_undegraded":"2024-04-28T12:39:13.276729+0000","last_fullsized":"2024-04-28T12:39:13.276729+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:18:19.496186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.357475+0000","last_change":"2024-04-28T12:39:10.099531+0000","last_active":"2024-04-28T12:39:12.357475+0000","last_peered":"2024-04-28T12:39:12.357475+0000","last_clean":"2024-04-28T12:39:12.357475+0000","last_became_active":"2024-04-28T12:39:10.099251+0000","last_became_peered":"2024-04-28T12:39:10.099251+0000","last_unstale":"2024-04-28T12:39:12.357475+0000","last_undegraded":"2024-04-28T12:39:12.357475+0000","last_fullsized":"2024-04-28T12:39:12.357475+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:14:41.434116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.277263+0000","last_change":"2024-04-28T12:39:11.112722+0000","last_active":"2024-04-28T12:39:13.277263+0000","last_peered":"2024-04-28T12:39:13.277263+0000","last_clean":"2024-04-28T12:39:13.277263+0000","last_became_active":"2024-04-28T12:39:11.109848+0000","last_became_peered":"2024-04-28T12:39:11.109848+0000","last_unstale":"2024-04-28T12:39:13.277263+0000","last_undegraded":"2024-04-28T12:39:13.277263+0000","last_fullsized":"2024-04-28T12:39:13.277263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:07:22.911705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.094622+0000","last_change":"2024-04-28T12:39:10.103341+0000","last_active":"2024-04-28T12:39:12.094622+0000","last_peered":"2024-04-28T12:39:12.094622+0000","last_clean":"2024-04-28T12:39:12.094622+0000","last_became_active":"2024-04-28T12:39:10.103061+0000","last_became_peered":"2024-04-28T12:39:10.103061+0000","last_unstale":"2024-04-28T12:39:12.094622+0000","last_undegraded":"2024-04-28T12:39:12.094622+0000","last_fullsized":"2024-04-28T12:39:12.094622+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:38:16.272036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.511559+0000","last_change":"2024-04-28T12:39:11.096078+0000","last_active":"2024-04-28T12:39:12.511559+0000","last_peered":"2024-04-28T12:39:12.511559+0000","last_clean":"2024-04-28T12:39:12.511559+0000","last_became_active":"2024-04-28T12:39:11.095775+0000","last_became_peered":"2024-04-28T12:39:11.095775+0000","last_unstale":"2024-04-28T12:39:12.511559+0000","last_undegraded":"2024-04-28T12:39:12.511559+0000","last_fullsized":"2024-04-28T12:39:12.511559+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:11:04.785848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105293+0000","last_change":"2024-04-28T12:39:10.082613+0000","last_active":"2024-04-28T12:39:13.105293+0000","last_peered":"2024-04-28T12:39:13.105293+0000","last_clean":"2024-04-28T12:39:13.105293+0000","last_became_active":"2024-04-28T12:39:10.082286+0000","last_became_peered":"2024-04-28T12:39:10.082286+0000","last_unstale":"2024-04-28T12:39:13.105293+0000","last_undegraded":"2024-04-28T12:39:13.105293+0000","last_fullsized":"2024-04-28T12:39:13.105293+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:41:23.038524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.105180+0000","last_change":"2024-04-28T12:39:11.101878+0000","last_active":"2024-04-28T12:39:13.105180+0000","last_peered":"2024-04-28T12:39:13.105180+0000","last_clean":"2024-04-28T12:39:13.105180+0000","last_became_active":"2024-04-28T12:39:11.096194+0000","last_became_peered":"2024-04-28T12:39:11.096194+0000","last_unstale":"2024-04-28T12:39:13.105180+0000","last_undegraded":"2024-04-28T12:39:13.105180+0000","last_fullsized":"2024-04-28T12:39:13.105180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:48:38.113586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.104858+0000","last_change":"2024-04-28T12:39:10.088421+0000","last_active":"2024-04-28T12:39:13.104858+0000","last_peered":"2024-04-28T12:39:13.104858+0000","last_clean":"2024-04-28T12:39:13.104858+0000","last_became_active":"2024-04-28T12:39:10.088119+0000","last_became_peered":"2024-04-28T12:39:10.088119+0000","last_unstale":"2024-04-28T12:39:13.104858+0000","last_undegraded":"2024-04-28T12:39:13.104858+0000","last_fullsized":"2024-04-28T12:39:13.104858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:18:45.075219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276263+0000","last_change":"2024-04-28T12:39:11.097561+0000","last_active":"2024-04-28T12:39:13.276263+0000","last_peered":"2024-04-28T12:39:13.276263+0000","last_clean":"2024-04-28T12:39:13.276263+0000","last_became_active":"2024-04-28T12:39:11.097323+0000","last_became_peered":"2024-04-28T12:39:11.097323+0000","last_unstale":"2024-04-28T12:39:13.276263+0000","last_undegraded":"2024-04-28T12:39:13.276263+0000","last_fullsized":"2024-04-28T12:39:13.276263+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:38:02.354280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.093990+0000","last_change":"2024-04-28T12:39:10.081244+0000","last_active":"2024-04-28T12:39:12.093990+0000","last_peered":"2024-04-28T12:39:12.093990+0000","last_clean":"2024-04-28T12:39:12.093990+0000","last_became_active":"2024-04-28T12:39:10.080943+0000","last_became_peered":"2024-04-28T12:39:10.080943+0000","last_unstale":"2024-04-28T12:39:12.093990+0000","last_undegraded":"2024-04-28T12:39:12.093990+0000","last_fullsized":"2024-04-28T12:39:12.093990+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:09:48.009392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.096789+0000","last_change":"2024-04-28T12:39:11.095515+0000","last_active":"2024-04-28T12:39:12.096789+0000","last_peered":"2024-04-28T12:39:12.096789+0000","last_clean":"2024-04-28T12:39:12.096789+0000","last_became_active":"2024-04-28T12:39:11.095255+0000","last_became_peered":"2024-04-28T12:39:11.095255+0000","last_unstale":"2024-04-28T12:39:12.096789+0000","last_undegraded":"2024-04-28T12:39:12.096789+0000","last_fullsized":"2024-04-28T12:39:12.096789+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T12:47:42.830968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.095556+0000","last_change":"2024-04-28T12:39:10.084354+0000","last_active":"2024-04-28T12:39:12.095556+0000","last_peered":"2024-04-28T12:39:12.095556+0000","last_clean":"2024-04-28T12:39:12.095556+0000","last_became_active":"2024-04-28T12:39:10.083623+0000","last_became_peered":"2024-04-28T12:39:10.083623+0000","last_unstale":"2024-04-28T12:39:12.095556+0000","last_undegraded":"2024-04-28T12:39:12.095556+0000","last_fullsized":"2024-04-28T12:39:12.095556+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:31:06.775995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.277196+0000","last_change":"2024-04-28T12:39:11.110940+0000","last_active":"2024-04-28T12:39:13.277196+0000","last_peered":"2024-04-28T12:39:13.277196+0000","last_clean":"2024-04-28T12:39:13.277196+0000","last_became_active":"2024-04-28T12:39:11.110533+0000","last_became_peered":"2024-04-28T12:39:11.110533+0000","last_unstale":"2024-04-28T12:39:13.277196+0000","last_undegraded":"2024-04-28T12:39:13.277196+0000","last_fullsized":"2024-04-28T12:39:13.277196+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:48:28.059974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.510450+0000","last_change":"2024-04-28T12:39:10.080505+0000","last_active":"2024-04-28T12:39:12.510450+0000","last_peered":"2024-04-28T12:39:12.510450+0000","last_clean":"2024-04-28T12:39:12.510450+0000","last_became_active":"2024-04-28T12:39:10.080178+0000","last_became_peered":"2024-04-28T12:39:10.080178+0000","last_unstale":"2024-04-28T12:39:12.510450+0000","last_undegraded":"2024-04-28T12:39:12.510450+0000","last_fullsized":"2024-04-28T12:39:12.510450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:07:17.221125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:12.356907+0000","last_change":"2024-04-28T12:39:11.108980+0000","last_active":"2024-04-28T12:39:12.356907+0000","last_peered":"2024-04-28T12:39:12.356907+0000","last_clean":"2024-04-28T12:39:12.356907+0000","last_became_active":"2024-04-28T12:39:11.108692+0000","last_became_peered":"2024-04-28T12:39:11.108692+0000","last_unstale":"2024-04-28T12:39:12.356907+0000","last_undegraded":"2024-04-28T12:39:12.356907+0000","last_fullsized":"2024-04-28T12:39:12.356907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:10.044250+0000","last_clean_scrub_stamp":"2024-04-28T12:39:10.044250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:07:42.536830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276974+0000","last_change":"2024-04-28T12:39:10.103090+0000","last_active":"2024-04-28T12:39:13.276974+0000","last_peered":"2024-04-28T12:39:13.276974+0000","last_clean":"2024-04-28T12:39:13.276974+0000","last_became_active":"2024-04-28T12:39:10.102773+0000","last_became_peered":"2024-04-28T12:39:10.102773+0000","last_unstale":"2024-04-28T12:39:13.276974+0000","last_undegraded":"2024-04-28T12:39:13.276974+0000","last_fullsized":"2024-04-28T12:39:13.276974+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:39:09.029756+0000","last_clean_scrub_stamp":"2024-04-28T12:39:09.029756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:08:12.923604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-28T12:39:13.276678+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:39:13.276678+0000","last_peered":"2024-04-28T12:39:13.276678+0000","last_clean":"2024-04-28T12:39:13.276678+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:39:13.276678+0000","last_undegraded":"2024-04-28T12:39:13.276678+0000","last_fullsized":"2024-04-28T12:39:13.276678+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":782086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":442368,"data_stored":190618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":446464,"data_stored":190148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":446464,"data_stored":190640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":191682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":190106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":190126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":780438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:39:20.994 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-28T12:39:20.994 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-28T12:39:21.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.245+0000 7f4660a00640 1 Processor -- start 2024-04-28T12:39:21.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.245+0000 7f4660a00640 1 -- start start 2024-04-28T12:39:21.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.247+0000 7f4660a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f465c15ee40 0x7f465c0d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.247+0000 7f4660a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f465c0d35d0 0x7f465c0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.247+0000 7f4660a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f465c0d04f0 con 0x7f465c0d57e0 2024-04-28T12:39:21.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.247+0000 7f4660a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f465c0d02a0 con 0x7f465c15ee40 2024-04-28T12:39:21.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.247+0000 7f4660a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f465c0b4110 con 0x7f465c0d35d0 2024-04-28T12:39:21.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.248+0000 7f465a800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f465c15ee40 0x7f465c0d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:21.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.248+0000 7f465b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f465c0d35d0 0x7f465c0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:21.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.248+0000 7f465b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f465c0d35d0 0x7f465c0b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:57042/0 (socket says 172.21.15.131:57042) 2024-04-28T12:39:21.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.248+0000 7f465a800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f465c15ee40 0x7f465c0d2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.131:34038/0 (socket says 172.21.15.131:34038) 2024-04-28T12:39:21.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.248+0000 7f465b200640 1 -- 172.21.15.131:0/915901561 learned_addr learned my addr 172.21.15.131:0/915901561 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:21.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.249+0000 7f465b200640 1 -- 172.21.15.131:0/915901561 >> v2:172.21.15.157:3300/0 conn(0x7f465c15ee40 msgr2=0x7f465c0d2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:21.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.249+0000 7f465b200640 1 --2- 172.21.15.131:0/915901561 >> v2:172.21.15.157:3300/0 conn(0x7f465c15ee40 0x7f465c0d2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.249+0000 7f465b200640 1 -- 172.21.15.131:0/915901561 >> v1:172.21.15.157:6789/0 conn(0x7f465c0d57e0 legacy=0x7f465c0d2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:21.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.249+0000 7f465b200640 1 -- 172.21.15.131:0/915901561 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465c0d07d0 con 0x7f465c0d35d0 2024-04-28T12:39:21.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.249+0000 7f465a800640 1 --2- 172.21.15.131:0/915901561 >> v2:172.21.15.157:3300/0 conn(0x7f465c15ee40 0x7f465c0d2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:39:21.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.251+0000 7f465b200640 1 --2- 172.21.15.131:0/915901561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f465c0d35d0 0x7f465c0b4860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4654009e50 tx=0x7f465402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81bb6ad567da312f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:21.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.251+0000 7f4659400640 1 -- 172.21.15.131:0/915901561 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654036070 con 0x7f465c0d35d0 2024-04-28T12:39:21.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.251+0000 7f4659400640 1 -- 172.21.15.131:0/915901561 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4654031600 con 0x7f465c0d35d0 2024-04-28T12:39:21.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.251+0000 7f4659400640 1 -- 172.21.15.131:0/915901561 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46540319d0 con 0x7f465c0d35d0 2024-04-28T12:39:21.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.252+0000 7f4660a00640 1 -- 172.21.15.131:0/915901561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f465c0d35d0 msgr2=0x7f465c0b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.252+0000 7f4660a00640 1 --2- 172.21.15.131:0/915901561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f465c0d35d0 0x7f465c0b4860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4654009e50 tx=0x7f465402a710 comp rx=0 tx=0).stop 2024-04-28T12:39:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.252+0000 7f4660a00640 1 -- 172.21.15.131:0/915901561 shutdown_connections 2024-04-28T12:39:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.252+0000 7f4660a00640 1 --2- 172.21.15.131:0/915901561 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f465c0d35d0 0x7f465c0b4860 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.252+0000 7f4660a00640 1 --2- 172.21.15.131:0/915901561 >> v2:172.21.15.157:3300/0 conn(0x7f465c15ee40 0x7f465c0d2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.252+0000 7f4660a00640 1 -- 172.21.15.131:0/915901561 >> 172.21.15.131:0/915901561 conn(0x7f465c0d4cc0 msgr2=0x7f465c0d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.252+0000 7f4660a00640 1 -- 172.21.15.131:0/915901561 shutdown_connections 2024-04-28T12:39:21.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.252+0000 7f4660a00640 1 -- 172.21.15.131:0/915901561 wait complete. 2024-04-28T12:39:21.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.254+0000 7f4660a00640 1 Processor -- start 2024-04-28T12:39:21.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.254+0000 7f4660a00640 1 -- start start 2024-04-28T12:39:21.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.255+0000 7f4660a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f465c0d35d0 0x7f465c1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.255+0000 7f4660a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f465c0d57e0 0x7f465c0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.255+0000 7f4660a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465c15ee40 0x7f465c0b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.255+0000 7f4660a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f465c15f220 con 0x7f465c15ee40 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.255+0000 7f4660a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f465c0b4360 con 0x7f465c0d57e0 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.255+0000 7f4660a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f465c15f3a0 con 0x7f465c0d35d0 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.255+0000 7f465b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465c15ee40 0x7f465c0b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.255+0000 7f465b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465c15ee40 0x7f465c0b8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56810/0 (socket says 172.21.15.131:56810) 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.255+0000 7f465b200640 1 -- 172.21.15.131:0/3263883884 learned_addr learned my addr 172.21.15.131:0/3263883884 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.256+0000 7f465a800640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f465c0d35d0 0x7f465c1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.256+0000 7f4659e00640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f465c0d57e0 0x7f465c0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.256+0000 7f465b200640 1 -- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f465c0d35d0 msgr2=0x7f465c1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.256+0000 7f465b200640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f465c0d35d0 0x7f465c1578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.256+0000 7f465b200640 1 -- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f465c0d57e0 msgr2=0x7f465c0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.256+0000 7f465b200640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f465c0d57e0 0x7f465c0cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.256+0000 7f465b200640 1 -- 172.21.15.131:0/3263883884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465c0b9fb0 con 0x7f465c15ee40 2024-04-28T12:39:21.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.256+0000 7f4659e00640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f465c0d57e0 0x7f465c0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:21.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.256+0000 7f465b200640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465c15ee40 0x7f465c0b8140 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4654009f70 tx=0x7f4654004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:21.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.257+0000 7f4652a00640 1 -- 172.21.15.131:0/3263883884 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465403f020 con 0x7f465c15ee40 2024-04-28T12:39:21.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.257+0000 7f4652a00640 1 -- 172.21.15.131:0/3263883884 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4654037040 con 0x7f465c15ee40 2024-04-28T12:39:21.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.257+0000 7f4660a00640 1 -- 172.21.15.131:0/3263883884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465c157de0 con 0x7f465c15ee40 2024-04-28T12:39:21.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.257+0000 7f4652a00640 1 -- 172.21.15.131:0/3263883884 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654036070 con 0x7f465c15ee40 2024-04-28T12:39:21.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.257+0000 7f4660a00640 1 -- 172.21.15.131:0/3263883884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f465c0ba230 con 0x7f465c15ee40 2024-04-28T12:39:21.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.258+0000 7f4660a00640 1 -- 172.21.15.131:0/3263883884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465c0506c0 con 0x7f465c15ee40 2024-04-28T12:39:21.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.259+0000 7f4652a00640 1 -- 172.21.15.131:0/3263883884 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4654042020 con 0x7f465c15ee40 2024-04-28T12:39:21.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.260+0000 7f4652a00640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f463c06c8f0 0x7f463c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.260+0000 7f4652a00640 1 -- 172.21.15.131:0/3263883884 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f46540ac130 con 0x7f465c15ee40 2024-04-28T12:39:21.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.261+0000 7f465a800640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f463c06c8f0 0x7f463c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.262+0000 7f465a800640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f463c06c8f0 0x7f463c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f464c009560 tx=0x7f464c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:21.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.265+0000 7f4652a00640 1 -- 172.21.15.131:0/3263883884 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4654049030 con 0x7f465c15ee40 2024-04-28T12:39:21.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.541+0000 7f4660a00640 1 -- 172.21.15.131:0/3263883884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f465c0cefd0 con 0x7f465c15ee40 2024-04-28T12:39:21.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.546+0000 7f4652a00640 1 -- 172.21.15.131:0/3263883884 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f465c0b9fb0 con 0x7f465c15ee40 2024-04-28T12:39:21.543 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-04-28T12:39:21.543 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDDQi5mElI+MxAAW17tKD+nhz2A3X8a4qZmbw== 2024-04-28T12:39:21.543 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-04-28T12:39:21.543 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-04-28T12:39:21.543 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-28T12:39:21.543 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-04-28T12:39:21.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.548+0000 7f4660a00640 1 -- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f463c06c8f0 msgr2=0x7f463c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:21.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.548+0000 7f4660a00640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f463c06c8f0 0x7f463c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f464c009560 tx=0x7f464c008040 comp rx=0 tx=0).stop 2024-04-28T12:39:21.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.548+0000 7f4660a00640 1 -- 172.21.15.131:0/3263883884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465c15ee40 msgr2=0x7f465c0b8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.548+0000 7f4660a00640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465c15ee40 0x7f465c0b8140 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4654009f70 tx=0x7f4654004a40 comp rx=0 tx=0).stop 2024-04-28T12:39:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.548+0000 7f4660a00640 1 -- 172.21.15.131:0/3263883884 shutdown_connections 2024-04-28T12:39:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.549+0000 7f4660a00640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f463c06c8f0 0x7f463c06edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.549+0000 7f4660a00640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465c15ee40 0x7f465c0b8140 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.549+0000 7f4660a00640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f465c0d57e0 0x7f465c0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.549+0000 7f4660a00640 1 --2- 172.21.15.131:0/3263883884 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f465c0d35d0 0x7f465c1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.549+0000 7f4660a00640 1 -- 172.21.15.131:0/3263883884 >> 172.21.15.131:0/3263883884 conn(0x7f465c0d4cc0 msgr2=0x7f465c0b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.549+0000 7f4660a00640 1 -- 172.21.15.131:0/3263883884 shutdown_connections 2024-04-28T12:39:21.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.549+0000 7f4660a00640 1 -- 172.21.15.131:0/3263883884 wait complete. 2024-04-28T12:39:21.564 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:39:21.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.767+0000 7fab18a00640 1 Processor -- start 2024-04-28T12:39:21.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.767+0000 7fab18a00640 1 -- start start 2024-04-28T12:39:21.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab18a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fab140afb70 0x7fab140cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab18a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab14072db0 con 0x7fab1415bb90 2024-04-28T12:39:21.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab18a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab140cee40 con 0x7fab140af710 2024-04-28T12:39:21.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab18a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fab140cec10 con 0x7fab140afb70 2024-04-28T12:39:21.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab13400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fab1415bb90 0x7fab140737a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36718/0 (socket says 172.21.15.131:36718) 2024-04-28T12:39:21.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab13400640 1 -- 172.21.15.131:0/1342413726 learned_addr learned my addr 172.21.15.131:0/1342413726 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:21.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab13e00640 1 --2- 172.21.15.131:0/1342413726 >> v2:172.21.15.157:3300/0 conn(0x7fab140afb70 0x7fab140cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:21.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 247625404 0 0) 0x7fab14072db0 con 0x7fab1415bb90 2024-04-28T12:39:21.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faaf8003520 con 0x7fab1415bb90 2024-04-28T12:39:21.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4169461568 0 0) 0x7faaf8003520 con 0x7fab1415bb90 2024-04-28T12:39:21.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 >> v2:172.21.15.157:3300/0 conn(0x7fab140afb70 msgr2=0x7fab140cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:21.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 --2- 172.21.15.131:0/1342413726 >> v2:172.21.15.157:3300/0 conn(0x7fab140afb70 0x7fab140cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 >> v1:172.21.15.157:6789/0 conn(0x7fab140af710 legacy=0x7fab140cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:21.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab140727f0 con 0x7fab1415bb90 2024-04-28T12:39:21.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fab04001670 con 0x7fab1415bb90 2024-04-28T12:39:21.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 >> v1:172.21.15.131:6789/0 conn(0x7fab1415bb90 legacy=0x7fab140737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:21.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 --2- 172.21.15.131:0/1342413726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faaf8003a00 0x7faaf8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 --2- 172.21.15.131:0/1342413726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf8004800 0x7faaf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 --2- 172.21.15.131:0/1342413726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaf8005210 0x7faaf800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faaf800c130 con 0x7faaf8004800 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faaf800c2e0 con 0x7faaf8005210 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faaf800c4e0 con 0x7faaf8003a00 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.769+0000 7fab0be00640 1 --2- 172.21.15.131:0/1342413726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf8004800 0x7faaf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.770+0000 7fab13e00640 1 --2- 172.21.15.131:0/1342413726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaf8005210 0x7faaf800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.770+0000 7fab13400640 1 --2- 172.21.15.131:0/1342413726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faaf8003a00 0x7faaf8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.770+0000 7fab0be00640 1 -- 172.21.15.131:0/1342413726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faaf8003a00 msgr2=0x7faaf8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.770+0000 7fab0be00640 1 --2- 172.21.15.131:0/1342413726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faaf8003a00 0x7faaf8003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.770+0000 7fab0be00640 1 -- 172.21.15.131:0/1342413726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaf8005210 msgr2=0x7faaf800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.770+0000 7fab0be00640 1 --2- 172.21.15.131:0/1342413726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faaf8005210 0x7faaf800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.770+0000 7fab0be00640 1 -- 172.21.15.131:0/1342413726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf8003520 con 0x7faaf8004800 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.770+0000 7fab13e00640 1 -- 172.21.15.131:0/1342413726 reap_dead start 2024-04-28T12:39:21.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.771+0000 7fab0be00640 1 --2- 172.21.15.131:0/1342413726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf8004800 0x7faaf8004c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faafc001750 tx=0x7faaf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a73c46142f6c099 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:21.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.771+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc001370 con 0x7faaf8004800 2024-04-28T12:39:21.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.771+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faafc0052d0 con 0x7faaf8004800 2024-04-28T12:39:21.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.771+0000 7fab12a00640 1 -- 172.21.15.131:0/1342413726 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc02abc0 con 0x7faaf8004800 2024-04-28T12:39:21.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.772+0000 7fab18a00640 1 -- 172.21.15.131:0/1342413726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf8004800 msgr2=0x7faaf8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:21.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.772+0000 7fab18a00640 1 --2- 172.21.15.131:0/1342413726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf8004800 0x7faaf8004c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faafc001750 tx=0x7faaf8005830 comp rx=0 tx=0).stop 2024-04-28T12:39:21.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.772+0000 7fab18a00640 1 -- 172.21.15.131:0/1342413726 shutdown_connections 2024-04-28T12:39:21.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.772+0000 7fab18a00640 1 --2- 172.21.15.131:0/1342413726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaf8004800 0x7faaf8004c80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.772+0000 7fab18a00640 1 -- 172.21.15.131:0/1342413726 >> 172.21.15.131:0/1342413726 conn(0x7fab14068090 msgr2=0x7fab14068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:21.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.772+0000 7fab18a00640 1 -- 172.21.15.131:0/1342413726 shutdown_connections 2024-04-28T12:39:21.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.772+0000 7fab18a00640 1 -- 172.21.15.131:0/1342413726 wait complete. 2024-04-28T12:39:21.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.774+0000 7fab18a00640 1 Processor -- start 2024-04-28T12:39:21.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.774+0000 7fab18a00640 1 -- start start 2024-04-28T12:39:21.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab18a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faaf8004800 0x7fab14146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab18a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab141517f0 0x7fab1415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab18a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab141f4ce0 0x7fab141f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab18a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fab140cf0e0 con 0x7fab141f4ce0 2024-04-28T12:39:21.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab18a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab140725a0 con 0x7fab141517f0 2024-04-28T12:39:21.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab18a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fab140b0000 con 0x7faaf8004800 2024-04-28T12:39:21.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab13e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab141f4ce0 0x7fab141f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:21.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab13e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab141f4ce0 0x7fab141f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56820/0 (socket says 172.21.15.131:56820) 2024-04-28T12:39:21.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab13e00640 1 -- 172.21.15.131:0/949304616 learned_addr learned my addr 172.21.15.131:0/949304616 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:21.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab13e00640 1 -- 172.21.15.131:0/949304616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faaf8004800 msgr2=0x7fab14146140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:21.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab13e00640 1 --2- 172.21.15.131:0/949304616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faaf8004800 0x7fab14146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab13e00640 1 -- 172.21.15.131:0/949304616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab141517f0 msgr2=0x7fab1415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:21.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab13e00640 1 --2- 172.21.15.131:0/949304616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab141517f0 0x7fab1415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:21.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.775+0000 7fab13e00640 1 -- 172.21.15.131:0/949304616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab14151db0 con 0x7fab141f4ce0 2024-04-28T12:39:21.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.776+0000 7fab13e00640 1 --2- 172.21.15.131:0/949304616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab141f4ce0 0x7fab141f5170 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009840 tx=0x7fab0c009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:21.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.776+0000 7fab0b400640 1 -- 172.21.15.131:0/949304616 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c007cc0 con 0x7fab141f4ce0 2024-04-28T12:39:21.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.776+0000 7fab0b400640 1 -- 172.21.15.131:0/949304616 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab0c019070 con 0x7fab141f4ce0 2024-04-28T12:39:21.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.776+0000 7fab18a00640 1 -- 172.21.15.131:0/949304616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab14152090 con 0x7fab141f4ce0 2024-04-28T12:39:21.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.776+0000 7fab0b400640 1 -- 172.21.15.131:0/949304616 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c007e60 con 0x7fab141f4ce0 2024-04-28T12:39:21.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.776+0000 7fab18a00640 1 -- 172.21.15.131:0/949304616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab14152650 con 0x7fab141f4ce0 2024-04-28T12:39:21.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.777+0000 7fab18a00640 1 -- 172.21.15.131:0/949304616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab14069350 con 0x7fab141f4ce0 2024-04-28T12:39:21.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.778+0000 7fab0b400640 1 -- 172.21.15.131:0/949304616 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab0c0126a0 con 0x7fab141f4ce0 2024-04-28T12:39:21.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.779+0000 7fab0b400640 1 --2- 172.21.15.131:0/949304616 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faaf006c920 0x7faaf006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:21.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.779+0000 7fab0b400640 1 -- 172.21.15.131:0/949304616 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fab0c08c200 con 0x7fab141f4ce0 2024-04-28T12:39:21.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.780+0000 7fab13400640 1 --2- 172.21.15.131:0/949304616 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faaf006c920 0x7faaf006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:21.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.782+0000 7fab13400640 1 --2- 172.21.15.131:0/949304616 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faaf006c920 0x7faaf006ede0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab04000c00 tx=0x7fab04009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:21.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:21.784+0000 7fab0b400640 1 -- 172.21.15.131:0/949304616 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab0c05a490 con 0x7fab141f4ce0 2024-04-28T12:39:22.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.028+0000 7fab18a00640 1 -- 172.21.15.131:0/949304616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab1406e050 con 0x7fab141f4ce0 2024-04-28T12:39:22.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.029+0000 7fab0b400640 1 -- 172.21.15.131:0/949304616 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5189 (secure 0 0 0) 0x7fab0c059d90 con 0x7fab141f4ce0 2024-04-28T12:39:22.026 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:22.026 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/361318591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":361318591},{"type":"v1","addr":"172.21.15.157:6836","nonce":361318591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/102335005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":102335005},{"type":"v1","addr":"172.21.15.131:6835","nonce":102335005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1609059776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1609059776},{"type":"v1","addr":"172.21.15.131:6833","nonce":1609059776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:39:10.533686+0000","modified":"2024-04-28T12:39:13.595067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34610":{"gid":34610,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/837933884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":837933884},{"type":"v1","addr":"172.21.15.157:6837","nonce":837933884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34610,"qdb_cluster":[34610]},"id":2}]} 2024-04-28T12:39:22.026 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2024-04-28T12:39:22.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.032+0000 7fab18a00640 1 -- 172.21.15.131:0/949304616 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faaf006c920 msgr2=0x7faaf006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.032+0000 7fab18a00640 1 --2- 172.21.15.131:0/949304616 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faaf006c920 0x7faaf006ede0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab04000c00 tx=0x7fab04009040 comp rx=0 tx=0).stop 2024-04-28T12:39:22.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.032+0000 7fab18a00640 1 -- 172.21.15.131:0/949304616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab141f4ce0 msgr2=0x7fab141f5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.032+0000 7fab18a00640 1 --2- 172.21.15.131:0/949304616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab141f4ce0 0x7fab141f5170 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009840 tx=0x7fab0c009d00 comp rx=0 tx=0).stop 2024-04-28T12:39:22.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.032+0000 7fab18a00640 1 -- 172.21.15.131:0/949304616 shutdown_connections 2024-04-28T12:39:22.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.032+0000 7fab18a00640 1 --2- 172.21.15.131:0/949304616 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faaf006c920 0x7faaf006ede0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.032+0000 7fab18a00640 1 --2- 172.21.15.131:0/949304616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab141f4ce0 0x7fab141f5170 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.032+0000 7fab18a00640 1 --2- 172.21.15.131:0/949304616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab141517f0 0x7fab1415e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.032+0000 7fab18a00640 1 --2- 172.21.15.131:0/949304616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faaf8004800 0x7fab14146140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.032+0000 7fab18a00640 1 -- 172.21.15.131:0/949304616 >> 172.21.15.131:0/949304616 conn(0x7fab14068090 msgr2=0x7fab1406e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:22.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.032+0000 7fab18a00640 1 -- 172.21.15.131:0/949304616 shutdown_connections 2024-04-28T12:39:22.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.033+0000 7fab18a00640 1 -- 172.21.15.131:0/949304616 wait complete. 2024-04-28T12:39:22.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-28T12:39:10.533686+0000', 'modified': '2024-04-28T12:39:13.595067+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34610}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34610': {'gid': 34610, 'name': 'd', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.157:6837/837933884', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6835', 'nonce': 837933884}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 837933884}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34610, 'qdb_cluster': [34610]} 2024-04-28T12:39:22.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-28T12:39:22.450 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-28T12:39:22.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.666+0000 7f4d2dc00640 1 Processor -- start 2024-04-28T12:39:22.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.666+0000 7f4d2dc00640 1 -- start start 2024-04-28T12:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.668+0000 7f4d2dc00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f4d280cf9b0 0x7f4d280b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.668+0000 7f4d2dc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d2815f2f0 con 0x7f4d2815ef10 2024-04-28T12:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.668+0000 7f4d2dc00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d2806e6c0 con 0x7f4d280b50b0 2024-04-28T12:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.668+0000 7f4d2dc00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f4d280cda40 con 0x7f4d280cf9b0 2024-04-28T12:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.668+0000 7f4d27e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4d2815ef10 0x7f4d2806de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36728/0 (socket says 172.21.15.131:36728) 2024-04-28T12:39:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d27e00640 1 -- 172.21.15.131:0/847599467 learned_addr learned my addr 172.21.15.131:0/847599467 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d2d200640 1 --2- 172.21.15.131:0/847599467 >> v2:172.21.15.157:3300/0 conn(0x7f4d280cf9b0 0x7f4d280b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 882915869 0 0) 0x7f4d2815f2f0 con 0x7f4d2815ef10 2024-04-28T12:39:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d0c003520 con 0x7f4d2815ef10 2024-04-28T12:39:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 143007093 0 0) 0x7f4d0c003520 con 0x7f4d2815ef10 2024-04-28T12:39:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 >> v2:172.21.15.157:3300/0 conn(0x7f4d280cf9b0 msgr2=0x7f4d280b7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d27400640 1 --2- 172.21.15.131:0/847599467 >> v2:172.21.15.157:3300/0 conn(0x7f4d280cf9b0 0x7f4d280b7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 >> v1:172.21.15.157:6789/0 conn(0x7f4d280b50b0 legacy=0x7f4d280cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d280cd110 con 0x7f4d2815ef10 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f4d18001670 con 0x7f4d2815ef10 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 >> v1:172.21.15.131:6789/0 conn(0x7f4d2815ef10 legacy=0x7f4d2806de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d27400640 1 --2- 172.21.15.131:0/847599467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d0c0039b0 0x7f4d0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.669+0000 7f4d27400640 1 --2- 172.21.15.131:0/847599467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0c004800 0x7f4d0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.670+0000 7f4d27400640 1 --2- 172.21.15.131:0/847599467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d0c0051c0 0x7f4d0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.670+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d280cda40 con 0x7f4d0c0051c0 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.670+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c00c110 con 0x7f4d0c0039b0 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.670+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0c00c310 con 0x7f4d0c004800 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.670+0000 7f4d2d200640 1 --2- 172.21.15.131:0/847599467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d0c0051c0 0x7f4d0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.670+0000 7f4d27e00640 1 --2- 172.21.15.131:0/847599467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d0c0039b0 0x7f4d0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.670+0000 7f4d2d200640 1 -- 172.21.15.131:0/847599467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0c004800 msgr2=0x7f4d0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.670+0000 7f4d2d200640 1 --2- 172.21.15.131:0/847599467 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0c004800 0x7f4d0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.670+0000 7f4d2d200640 1 -- 172.21.15.131:0/847599467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d0c0039b0 msgr2=0x7f4d0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.670+0000 7f4d2d200640 1 --2- 172.21.15.131:0/847599467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d0c0039b0 0x7f4d0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.670+0000 7f4d2d200640 1 -- 172.21.15.131:0/847599467 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c003520 con 0x7f4d0c0051c0 2024-04-28T12:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.671+0000 7f4d2d200640 1 -- 172.21.15.131:0/847599467 reap_dead start 2024-04-28T12:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.671+0000 7f4d2d200640 1 --2- 172.21.15.131:0/847599467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d0c0051c0 0x7f4d0c00b9c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4d20007330 tx=0x7f4d0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb58df51420bd1a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.671+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20009be0 con 0x7f4d0c0051c0 2024-04-28T12:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.672+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d20003b90 con 0x7f4d0c0051c0 2024-04-28T12:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.672+0000 7f4d27400640 1 -- 172.21.15.131:0/847599467 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20004020 con 0x7f4d0c0051c0 2024-04-28T12:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.672+0000 7f4d2dc00640 1 -- 172.21.15.131:0/847599467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d0c0051c0 msgr2=0x7f4d0c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.672+0000 7f4d2dc00640 1 --2- 172.21.15.131:0/847599467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d0c0051c0 0x7f4d0c00b9c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4d20007330 tx=0x7f4d0c005830 comp rx=0 tx=0).stop 2024-04-28T12:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.672+0000 7f4d2dc00640 1 -- 172.21.15.131:0/847599467 shutdown_connections 2024-04-28T12:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.672+0000 7f4d2dc00640 1 --2- 172.21.15.131:0/847599467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d0c0051c0 0x7f4d0c00b9c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.672+0000 7f4d2dc00640 1 -- 172.21.15.131:0/847599467 >> 172.21.15.131:0/847599467 conn(0x7f4d280b4590 msgr2=0x7f4d280b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.672+0000 7f4d2dc00640 1 -- 172.21.15.131:0/847599467 shutdown_connections 2024-04-28T12:39:22.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.672+0000 7f4d2dc00640 1 -- 172.21.15.131:0/847599467 wait complete. 2024-04-28T12:39:22.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.674+0000 7f4d2dc00640 1 Processor -- start 2024-04-28T12:39:22.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.674+0000 7f4d2dc00640 1 -- start start 2024-04-28T12:39:22.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.675+0000 7f4d2dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0c0051c0 0x7f4d2815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.675+0000 7f4d2dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d280c87e0 0x7f4d280c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.675+0000 7f4d2dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d280c6510 0x7f4d280c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.675+0000 7f4d2dc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d280b5890 con 0x7f4d280c6510 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.675+0000 7f4d2dc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d280cd6d0 con 0x7f4d280c87e0 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.675+0000 7f4d2dc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d28160700 con 0x7f4d0c0051c0 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.675+0000 7f4d2d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d280c6510 0x7f4d280c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.676+0000 7f4d2d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d280c6510 0x7f4d280c6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56832/0 (socket says 172.21.15.131:56832) 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.676+0000 7f4d2d200640 1 -- 172.21.15.131:0/2192137571 learned_addr learned my addr 172.21.15.131:0/2192137571 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.676+0000 7f4d27e00640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0c0051c0 0x7f4d2815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.676+0000 7f4d1fe00640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d280c87e0 0x7f4d280c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.676+0000 7f4d2d200640 1 -- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0c0051c0 msgr2=0x7f4d2815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.676+0000 7f4d2d200640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0c0051c0 0x7f4d2815c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.676+0000 7f4d2d200640 1 -- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d280c87e0 msgr2=0x7f4d280c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.676+0000 7f4d2d200640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d280c87e0 0x7f4d280c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.676+0000 7f4d2d200640 1 -- 172.21.15.131:0/2192137571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2815fe10 con 0x7f4d280c6510 2024-04-28T12:39:22.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.676+0000 7f4d2d200640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d280c6510 0x7f4d280c6970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4d2002b070 tx=0x7f4d20004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:22.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.677+0000 7f4d24c00640 1 -- 172.21.15.131:0/2192137571 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20045070 con 0x7f4d280c6510 2024-04-28T12:39:22.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.677+0000 7f4d24c00640 1 -- 172.21.15.131:0/2192137571 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d20003990 con 0x7f4d280c6510 2024-04-28T12:39:22.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.677+0000 7f4d24c00640 1 -- 172.21.15.131:0/2192137571 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20009be0 con 0x7f4d280c6510 2024-04-28T12:39:22.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.677+0000 7f4d2dc00640 1 -- 172.21.15.131:0/2192137571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d281600b0 con 0x7f4d280c6510 2024-04-28T12:39:22.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.677+0000 7f4d2dc00640 1 -- 172.21.15.131:0/2192137571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d280c91a0 con 0x7f4d280c6510 2024-04-28T12:39:22.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.679+0000 7f4d24c00640 1 -- 172.21.15.131:0/2192137571 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d20048070 con 0x7f4d280c6510 2024-04-28T12:39:22.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.680+0000 7f4d24c00640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d0406c8f0 0x7f4d0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.680+0000 7f4d24c00640 1 -- 172.21.15.131:0/2192137571 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f4d200ac180 con 0x7f4d280c6510 2024-04-28T12:39:22.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.681+0000 7f4d32170740 1 Processor -- start 2024-04-28T12:39:22.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.681+0000 7f4d27e00640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d0406c8f0 0x7f4d0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.681+0000 7f4d32170740 1 -- start start 2024-04-28T12:39:22.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.682+0000 7f4d32170740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 0x556646906910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.682+0000 7f4d32170740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556646940e80 0x556646904b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.682+0000 7f4d32170740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.682+0000 7f4d32170740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556646868600 con 0x55664692f270 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.682+0000 7f4d32170740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556646805100 con 0x556646940e80 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.682+0000 7f4d32170740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5566468645c0 con 0x5566469047b0 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.682+0000 7f4d27e00640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d0406c8f0 0x7f4d0406edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d18000fb0 tx=0x7f4d18009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.682+0000 7f4d2d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.682+0000 7f4d2d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56840/0 (socket says 172.21.15.131:56840) 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.682+0000 7f4d2d200640 1 -- 172.21.15.131:0/1588884648 learned_addr learned my addr 172.21.15.131:0/1588884648 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.682+0000 7f4d27e00640 1 --2- 172.21.15.131:0/1588884648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 0x556646906910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.683+0000 7f4d1fe00640 1 --2- 172.21.15.131:0/1588884648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556646940e80 0x556646904b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.683+0000 7f4d2d200640 1 -- 172.21.15.131:0/1588884648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 msgr2=0x556646906910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.683+0000 7f4d2d200640 1 --2- 172.21.15.131:0/1588884648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 0x556646906910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.683+0000 7f4d2d200640 1 -- 172.21.15.131:0/1588884648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556646940e80 msgr2=0x556646904b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.683+0000 7f4d2d200640 1 --2- 172.21.15.131:0/1588884648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556646940e80 0x556646904b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.683+0000 7f4d2d200640 1 -- 172.21.15.131:0/1588884648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664657a2d0 con 0x55664692f270 2024-04-28T12:39:22.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.683+0000 7f4d1fe00640 1 --2- 172.21.15.131:0/1588884648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556646940e80 0x556646904b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:22.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.683+0000 7f4d2d200640 1 --2- 172.21.15.131:0/1588884648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692f6f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4d200aa420 tx=0x7f4d20003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbe21a6d8308e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:22.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.683+0000 7f4d1cc00640 1 -- 172.21.15.131:0/1588884648 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20079da0 con 0x55664692f270 2024-04-28T12:39:22.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.683+0000 7f4d1cc00640 1 -- 172.21.15.131:0/1588884648 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2007e150 con 0x55664692f270 2024-04-28T12:39:22.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.683+0000 7f4d1cc00640 1 -- 172.21.15.131:0/1588884648 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20082980 con 0x55664692f270 2024-04-28T12:39:22.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.684+0000 7f4d32170740 1 -- 172.21.15.131:0/1588884648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 msgr2=0x55664692f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.684+0000 7f4d32170740 1 --2- 172.21.15.131:0/1588884648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692f6f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4d200aa420 tx=0x7f4d20003b60 comp rx=0 tx=0).stop 2024-04-28T12:39:22.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.684+0000 7f4d32170740 1 -- 172.21.15.131:0/1588884648 shutdown_connections 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.684+0000 7f4d32170740 1 --2- 172.21.15.131:0/1588884648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692f6f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.684+0000 7f4d32170740 1 --2- 172.21.15.131:0/1588884648 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556646940e80 0x556646904b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.684+0000 7f4d32170740 1 --2- 172.21.15.131:0/1588884648 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 0x556646906910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.684+0000 7f4d32170740 1 -- 172.21.15.131:0/1588884648 >> 172.21.15.131:0/1588884648 conn(0x556646898d10 msgr2=0x55664694a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.684+0000 7f4d32170740 1 -- 172.21.15.131:0/1588884648 shutdown_connections 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.684+0000 7f4d32170740 1 -- 172.21.15.131:0/1588884648 wait complete. 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.685+0000 7f4d32170740 1 -- start start 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.685+0000 7f4d32170740 1 Processor -- start 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.686+0000 7f4d32170740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 0x55664691e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.686+0000 7f4d32170740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664692f270 0x55664691c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.686+0000 7f4d32170740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556646940e80 0x55664691b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.686+0000 7f4d32170740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556646805100 con 0x556646940e80 2024-04-28T12:39:22.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.686+0000 7f4d32170740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5566467eff70 con 0x55664692f270 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.686+0000 7f4d32170740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x556646868600 con 0x5566469047b0 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.686+0000 7f4d32170740 10 client.0 _ll_register_callbacks cb 0x55664685e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.686+0000 7f4d2d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556646940e80 0x55664691b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.686+0000 7f4d2d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556646940e80 0x55664691b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56854/0 (socket says 172.21.15.131:56854) 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.686+0000 7f4d2d200640 1 -- 172.21.15.131:0/1823535795 learned_addr learned my addr 172.21.15.131:0/1823535795 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d27e00640 1 --2- 172.21.15.131:0/1823535795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 0x55664691e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d2d200640 1 -- 172.21.15.131:0/1823535795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 msgr2=0x55664691e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d2d200640 1 --2- 172.21.15.131:0/1823535795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 0x55664691e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d2d200640 1 -- 172.21.15.131:0/1823535795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664692f270 msgr2=0x55664691c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d2d200640 1 --2- 172.21.15.131:0/1823535795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664692f270 0x55664691c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d2d200640 1 -- 172.21.15.131:0/1823535795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566466446b0 con 0x556646940e80 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d2d200640 1 --2- 172.21.15.131:0/1823535795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556646940e80 0x55664691b3b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4d20079aa0 tx=0x7f4d20009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4adc82547aa23ed server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d15600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-28T12:39:22.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d15600640 1 -- 172.21.15.131:0/1823535795 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20079da0 con 0x556646940e80 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d15600640 1 -- 172.21.15.131:0/1823535795 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f4d200aa120 con 0x556646940e80 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d32170740 1 -- 172.21.15.131:0/1823535795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d2007e150 con 0x556646940e80 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d15600640 1 -- 172.21.15.131:0/1823535795 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4d20092170 con 0x556646940e80 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d32170740 10 client.5691 fetch_fsmap learned FSMap version 17 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.687+0000 7f4d32170740 1 -- 172.21.15.131:0/1823535795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55664691d3a0 con 0x556646940e80 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.688+0000 7f4d15600640 1 -- 172.21.15.131:0/1823535795 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2007e150 con 0x556646940e80 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.688+0000 7f4d15600640 1 -- 172.21.15.131:0/1823535795 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f4d20082bd0 con 0x556646940e80 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.688+0000 7f4d32170740 10 client.5691 fetch_fsmap finished waiting for FSMap version 17 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.688+0000 7f4d32170740 10 client.5691 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.688+0000 7f4d32170740 20 client.5691 populate_metadata read hostname 'smithi131' 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.688+0000 7f4d32170740 1 --2- 172.21.15.131:0/1823535795 >> [v2:172.21.15.157:6835/837933884,v1:172.21.15.157:6837/837933884] conn(0x556646922310 0x55664691b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.688+0000 7f4d32170740 4 client.5691 mds_command: new command op to 34610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:39:22.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.688+0000 7f4d32170740 1 -- 172.21.15.131:0/1823535795 --> [v2:172.21.15.157:6835/837933884,v1:172.21.15.157:6837/837933884] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556646876210 con 0x556646922310 2024-04-28T12:39:22.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.689+0000 7f4d27e00640 1 --2- 172.21.15.131:0/1823535795 >> [v2:172.21.15.157:6835/837933884,v1:172.21.15.157:6837/837933884] conn(0x556646922310 0x55664691b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.690+0000 7f4d27e00640 1 --2- 172.21.15.131:0/1823535795 >> [v2:172.21.15.157:6835/837933884,v1:172.21.15.157:6837/837933884] conn(0x556646922310 0x55664691b8f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f1535501674e26 server_cookie=29d576045318d75d in_seq=0 out_seq=0 2024-04-28T12:39:22.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.690+0000 7f4d15600640 10 client.5691 ms_handle_connect on v2:172.21.15.157:6835/837933884 2024-04-28T12:39:22.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.692+0000 7f4d15600640 1 -- 172.21.15.131:0/1823535795 <== mds.0 v2:172.21.15.157:6835/837933884 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556646876210 con 0x556646922310 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.692+0000 7f4d15600640 10 client.5691 handle_command_reply: tid=0 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.692+0000 7f4d32170740 1 client.5691 shutdown 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.693+0000 7f4d32170740 1 -- 172.21.15.131:0/1823535795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556646940e80 msgr2=0x55664691b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.693+0000 7f4d32170740 1 --2- 172.21.15.131:0/1823535795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556646940e80 0x55664691b3b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4d20079aa0 tx=0x7f4d20009760 comp rx=0 tx=0).stop 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.693+0000 7f4d32170740 1 -- 172.21.15.131:0/1823535795 shutdown_connections 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.693+0000 7f4d32170740 1 --2- 172.21.15.131:0/1823535795 >> [v2:172.21.15.157:6835/837933884,v1:172.21.15.157:6837/837933884] conn(0x556646922310 0x55664691b8f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.693+0000 7f4d32170740 1 --2- 172.21.15.131:0/1823535795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556646940e80 0x55664691b3b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.693+0000 7f4d32170740 1 --2- 172.21.15.131:0/1823535795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55664692f270 0x55664691c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.693+0000 7f4d32170740 1 --2- 172.21.15.131:0/1823535795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 0x55664691e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.693+0000 7f4d32170740 1 -- 172.21.15.131:0/1823535795 >> 172.21.15.131:0/1823535795 conn(0x556646898d10 msgr2=0x55664694d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.693+0000 7f4d15600640 0 client.5691 ms_handle_reset on v2:172.21.15.157:6835/837933884 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.693+0000 7f4d32170740 1 -- 172.21.15.131:0/1823535795 shutdown_connections 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.693+0000 7f4d32170740 1 -- 172.21.15.131:0/1823535795 wait complete. 2024-04-28T12:39:22.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.693+0000 7f4d32170740 20 client.5691 trim_cache size 0 max 16384 2024-04-28T12:39:22.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.714+0000 7f4d32170740 1 Processor -- start 2024-04-28T12:39:22.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.714+0000 7f4d32170740 1 -- start start 2024-04-28T12:39:22.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.715+0000 7f4d32170740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 0x5566469316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.715+0000 7f4d32170740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.715+0000 7f4d32170740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556646940e80 0x55664692d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.715+0000 7f4d32170740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d282011b0 con 0x55664692f270 2024-04-28T12:39:22.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.715+0000 7f4d32170740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28200610 con 0x556646940e80 2024-04-28T12:39:22.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.715+0000 7f4d32170740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d28200b90 con 0x5566469047b0 2024-04-28T12:39:22.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.716+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.716+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56868/0 (socket says 172.21.15.131:56868) 2024-04-28T12:39:22.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.716+0000 7f4d1fe00640 1 -- 172.21.15.131:0/2966789738 learned_addr learned my addr 172.21.15.131:0/2966789738 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.716+0000 7f4d2d200640 1 --2- 172.21.15.131:0/2966789738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556646940e80 0x55664692d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.716+0000 7f4d27e00640 1 --2- 172.21.15.131:0/2966789738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 0x5566469316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.716+0000 7f4d1fe00640 1 -- 172.21.15.131:0/2966789738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 msgr2=0x5566469316c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.716+0000 7f4d1fe00640 1 --2- 172.21.15.131:0/2966789738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 0x5566469316c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.716+0000 7f4d1fe00640 1 -- 172.21.15.131:0/2966789738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556646940e80 msgr2=0x55664692d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.716+0000 7f4d1fe00640 1 --2- 172.21.15.131:0/2966789738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556646940e80 0x55664692d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.716+0000 7f4d1fe00640 1 -- 172.21.15.131:0/2966789738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556646933470 con 0x55664692f270 2024-04-28T12:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.716+0000 7f4d2d200640 1 --2- 172.21.15.131:0/2966789738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556646940e80 0x55664692d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.716+0000 7f4d1fe00640 1 --2- 172.21.15.131:0/2966789738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692c930 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4d10001670 tx=0x7f4d10001b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65db74c03e2fb039 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.717+0000 7f4d17e00640 1 -- 172.21.15.131:0/2966789738 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d100052e0 con 0x55664692f270 2024-04-28T12:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.717+0000 7f4d17e00640 1 -- 172.21.15.131:0/2966789738 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d10018070 con 0x55664692f270 2024-04-28T12:39:22.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.717+0000 7f4d17e00640 1 -- 172.21.15.131:0/2966789738 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000f9f0 con 0x55664692f270 2024-04-28T12:39:22.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.717+0000 7f4d32170740 1 -- 172.21.15.131:0/2966789738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 msgr2=0x55664692c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.717+0000 7f4d32170740 1 --2- 172.21.15.131:0/2966789738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692c930 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4d10001670 tx=0x7f4d10001b30 comp rx=0 tx=0).stop 2024-04-28T12:39:22.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.717+0000 7f4d32170740 1 -- 172.21.15.131:0/2966789738 shutdown_connections 2024-04-28T12:39:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.717+0000 7f4d32170740 1 --2- 172.21.15.131:0/2966789738 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x556646940e80 0x55664692d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.717+0000 7f4d32170740 1 --2- 172.21.15.131:0/2966789738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692c930 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.717+0000 7f4d32170740 1 --2- 172.21.15.131:0/2966789738 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5566469047b0 0x5566469316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.717+0000 7f4d32170740 1 -- 172.21.15.131:0/2966789738 >> 172.21.15.131:0/2966789738 conn(0x556646898d10 msgr2=0x556646932d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.717+0000 7f4d32170740 1 -- 172.21.15.131:0/2966789738 shutdown_connections 2024-04-28T12:39:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.717+0000 7f4d32170740 1 -- 172.21.15.131:0/2966789738 wait complete. 2024-04-28T12:39:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.718+0000 7f4d32170740 1 -- start start 2024-04-28T12:39:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.718+0000 7f4d32170740 1 Processor -- start 2024-04-28T12:39:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d32170740 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566469047b0 0x55664694d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d32170740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d32170740 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556646940e80 0x55664692ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d32170740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556646805100 con 0x55664692f270 2024-04-28T12:39:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d32170740 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x556646868600 con 0x5566469047b0 2024-04-28T12:39:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d32170740 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5566467eff70 con 0x556646940e80 2024-04-28T12:39:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d32170740 10 client.0 _ll_register_callbacks cb 0x7f4d20091070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:39:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56874/0 (socket says 172.21.15.131:56874) 2024-04-28T12:39:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d1fe00640 1 -- 172.21.15.131:0/3435098202 learned_addr learned my addr 172.21.15.131:0/3435098202 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d27e00640 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566469047b0 0x55664694d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d1fe00640 1 -- 172.21.15.131:0/3435098202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556646940e80 msgr2=0x55664692ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d1fe00640 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556646940e80 0x55664692ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d1fe00640 1 -- 172.21.15.131:0/3435098202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566469047b0 msgr2=0x55664694d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d1fe00640 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566469047b0 0x55664694d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d1fe00640 1 -- 172.21.15.131:0/3435098202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664694e430 con 0x55664692f270 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.720+0000 7f4d27e00640 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566469047b0 0x55664694d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.721+0000 7f4d1fe00640 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692c930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4d10000f20 tx=0x7f4d10007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbd4ede5815fe28 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.721+0000 7f4d15600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.721+0000 7f4d15600640 1 -- 172.21.15.131:0/3435098202 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10007690 con 0x55664692f270 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.721+0000 7f4d15600640 1 -- 172.21.15.131:0/3435098202 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f4d100067d0 con 0x55664692f270 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.721+0000 7f4d32170740 1 -- 172.21.15.131:0/3435098202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d10018070 con 0x55664692f270 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.721+0000 7f4d15600640 1 -- 172.21.15.131:0/3435098202 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4d1002d050 con 0x55664692f270 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.721+0000 7f4d32170740 10 client.5697 fetch_fsmap learned FSMap version 17 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.721+0000 7f4d32170740 1 -- 172.21.15.131:0/3435098202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5566469aa5b0 con 0x55664692f270 2024-04-28T12:39:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.722+0000 7f4d15600640 1 -- 172.21.15.131:0/3435098202 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d10018070 con 0x55664692f270 2024-04-28T12:39:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.722+0000 7f4d15600640 1 -- 172.21.15.131:0/3435098202 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f4d10030020 con 0x55664692f270 2024-04-28T12:39:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.722+0000 7f4d32170740 10 client.5697 fetch_fsmap finished waiting for FSMap version 17 2024-04-28T12:39:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.722+0000 7f4d32170740 10 client.5697 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-28T12:39:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.722+0000 7f4d32170740 20 client.5697 populate_metadata read hostname 'smithi131' 2024-04-28T12:39:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.722+0000 7f4d32170740 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.157:6835/837933884,v1:172.21.15.157:6837/837933884] conn(0x5566469c4fd0 0x55664693fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.722+0000 7f4d32170740 4 client.5697 mds_command: new command op to 34610 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:39:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.722+0000 7f4d32170740 1 -- 172.21.15.131:0/3435098202 --> [v2:172.21.15.157:6835/837933884,v1:172.21.15.157:6837/837933884] -- command(tid 0: {"prefix": "status"}) -- 0x556646870080 con 0x5566469c4fd0 2024-04-28T12:39:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.722+0000 7f4d27e00640 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.157:6835/837933884,v1:172.21.15.157:6837/837933884] conn(0x5566469c4fd0 0x55664693fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.723+0000 7f4d27e00640 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.157:6835/837933884,v1:172.21.15.157:6837/837933884] conn(0x5566469c4fd0 0x55664693fc00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a546f38b15253de7 server_cookie=170227ecc10e2ac3 in_seq=0 out_seq=0 2024-04-28T12:39:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.723+0000 7f4d15600640 10 client.5697 ms_handle_connect on v2:172.21.15.157:6835/837933884 2024-04-28T12:39:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.724+0000 7f4d15600640 1 -- 172.21.15.131:0/3435098202 <== mds.0 v2:172.21.15.157:6835/837933884 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x556646870080 con 0x5566469c4fd0 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.724+0000 7f4d15600640 10 client.5697 handle_command_reply: tid=0 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.724+0000 7f4d32170740 1 client.5697 shutdown 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.724+0000 7f4d32170740 1 -- 172.21.15.131:0/3435098202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 msgr2=0x55664692c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.724+0000 7f4d32170740 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692c930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4d10000f20 tx=0x7f4d10007b50 comp rx=0 tx=0).stop 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.724+0000 7f4d32170740 1 -- 172.21.15.131:0/3435098202 shutdown_connections 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.724+0000 7f4d32170740 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.157:6835/837933884,v1:172.21.15.157:6837/837933884] conn(0x5566469c4fd0 0x55664693fc00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.724+0000 7f4d32170740 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x556646940e80 0x55664692ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.724+0000 7f4d32170740 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55664692f270 0x55664692c930 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.724+0000 7f4d32170740 1 --2- 172.21.15.131:0/3435098202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5566469047b0 0x55664694d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.724+0000 7f4d15600640 0 client.5697 ms_handle_reset on v2:172.21.15.157:6835/837933884 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.724+0000 7f4d32170740 1 -- 172.21.15.131:0/3435098202 >> 172.21.15.131:0/3435098202 conn(0x556646898d10 msgr2=0x55664696cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d32170740 1 -- 172.21.15.131:0/3435098202 shutdown_connections 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d32170740 1 -- 172.21.15.131:0/3435098202 wait complete. 2024-04-28T12:39:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d32170740 20 client.5697 trim_cache size 0 max 16384 2024-04-28T12:39:22.724 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:39:22.724 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "fc0efa02-d382-4f6a-947e-cbe7ab2d58d1", 2024-04-28T12:39:22.724 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-04-28T12:39:22.724 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 34610, 2024-04-28T12:39:22.724 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-04-28T12:39:22.724 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-04-28T12:39:22.724 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-04-28T12:39:22.724 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 11.657896957, 2024-04-28T12:39:22.724 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 17, 2024-04-28T12:39:22.724 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 38, 2024-04-28T12:39:22.724 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 38, 2024-04-28T12:39:22.725 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 20.467240095000001 2024-04-28T12:39:22.725 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:39:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d2dc00640 1 -- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d0406c8f0 msgr2=0x7f4d0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d2dc00640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d0406c8f0 0x7f4d0406edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d18000fb0 tx=0x7f4d18009040 comp rx=0 tx=0).stop 2024-04-28T12:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d2dc00640 1 -- 172.21.15.131:0/2192137571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d280c6510 msgr2=0x7f4d280c6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d2dc00640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d280c6510 0x7f4d280c6970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4d2002b070 tx=0x7f4d20004ab0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d2dc00640 1 -- 172.21.15.131:0/2192137571 shutdown_connections 2024-04-28T12:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d2dc00640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f4d0406c8f0 0x7f4d0406edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d2dc00640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d280c6510 0x7f4d280c6970 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d2dc00640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d280c87e0 0x7f4d280c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d2dc00640 1 --2- 172.21.15.131:0/2192137571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d0c0051c0 0x7f4d2815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.725+0000 7f4d2dc00640 1 -- 172.21.15.131:0/2192137571 >> 172.21.15.131:0/2192137571 conn(0x7f4d280b4590 msgr2=0x7f4d280b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.726+0000 7f4d2dc00640 1 -- 172.21.15.131:0/2192137571 shutdown_connections 2024-04-28T12:39:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.726+0000 7f4d2dc00640 1 -- 172.21.15.131:0/2192137571 wait complete. 2024-04-28T12:39:22.741 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-28T12:39:22.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.948+0000 7fe9b2400640 1 Processor -- start 2024-04-28T12:39:22.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.949+0000 7fe9b2400640 1 -- start start 2024-04-28T12:39:22.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.950+0000 7fe9b2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ac0d3480 0x7fe9ac0c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.950+0000 7fe9b2400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe9ac162880 0x7fe9ac0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.950+0000 7fe9b2400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ac0cf860 con 0x7fe9ac0d1d90 2024-04-28T12:39:22.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.950+0000 7fe9b2400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe9ac0d3090 con 0x7fe9ac162880 2024-04-28T12:39:22.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.950+0000 7fe9b2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac0d5b50 con 0x7fe9ac0d3480 2024-04-28T12:39:22.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.950+0000 7fe9a3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe9ac0d1d90 0x7fe9ac162170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36736/0 (socket says 172.21.15.131:36736) 2024-04-28T12:39:22.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.950+0000 7fe9a3e00640 1 -- 172.21.15.131:0/330422249 learned_addr learned my addr 172.21.15.131:0/330422249 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:39:22.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.950+0000 7fe9b1000640 1 --2- 172.21.15.131:0/330422249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ac0d3480 0x7fe9ac0c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.950+0000 7fe9b1a00640 1 --2- 172.21.15.131:0/330422249 >> v2:172.21.15.157:3300/0 conn(0x7fe9ac162880 0x7fe9ac0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2557695076 0 0) 0x7fe9ac0cf860 con 0x7fe9ac0d1d90 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe990003520 con 0x7fe9ac0d1d90 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1960609479 0 0) 0x7fe990003520 con 0x7fe9ac0d1d90 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ac0d3480 msgr2=0x7fe9ac0c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 --2- 172.21.15.131:0/330422249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ac0d3480 0x7fe9ac0c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 >> v2:172.21.15.157:3300/0 conn(0x7fe9ac162880 msgr2=0x7fe9ac0cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 --2- 172.21.15.131:0/330422249 >> v2:172.21.15.157:3300/0 conn(0x7fe9ac162880 0x7fe9ac0cffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac0cfb00 con 0x7fe9ac0d1d90 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fe994001670 con 0x7fe9ac0d1d90 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 >> v1:172.21.15.131:6789/0 conn(0x7fe9ac0d1d90 legacy=0x7fe9ac162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 --2- 172.21.15.131:0/330422249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ac0d3480 0x7fe9900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 --2- 172.21.15.131:0/330422249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe990003fe0 0x7fe990004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 --2- 172.21.15.131:0/330422249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe990004d40 0x7fe9900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac0d3090 con 0x7fe990003fe0 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac0d5b50 con 0x7fe990004d40 2024-04-28T12:39:22.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe99000bac0 con 0x7fe9ac0d3480 2024-04-28T12:39:22.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.951+0000 7fe9a3e00640 1 --2- 172.21.15.131:0/330422249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe990003fe0 0x7fe990004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.952+0000 7fe9b1a00640 1 --2- 172.21.15.131:0/330422249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe990004d40 0x7fe9900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.952+0000 7fe9a3e00640 1 -- 172.21.15.131:0/330422249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ac0d3480 msgr2=0x7fe9900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.952+0000 7fe9a3e00640 1 --2- 172.21.15.131:0/330422249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9ac0d3480 0x7fe9900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.952+0000 7fe9a3e00640 1 -- 172.21.15.131:0/330422249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe990004d40 msgr2=0x7fe9900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:39:22.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.952+0000 7fe9a3e00640 1 --2- 172.21.15.131:0/330422249 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe990004d40 0x7fe9900051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.952+0000 7fe9a3e00640 1 -- 172.21.15.131:0/330422249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe990003520 con 0x7fe990003fe0 2024-04-28T12:39:22.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.953+0000 7fe9a3e00640 1 --2- 172.21.15.131:0/330422249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe990003fe0 0x7fe990004800 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe994007fd0 tx=0x7fe990005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2c9b806e7b886c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:22.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.953+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994007d50 con 0x7fe990003fe0 2024-04-28T12:39:22.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.953+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe994005af0 con 0x7fe990003fe0 2024-04-28T12:39:22.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.953+0000 7fe9abe00640 1 -- 172.21.15.131:0/330422249 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994007070 con 0x7fe990003fe0 2024-04-28T12:39:22.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.954+0000 7fe9b2400640 1 -- 172.21.15.131:0/330422249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe990003fe0 msgr2=0x7fe990004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.954+0000 7fe9b2400640 1 --2- 172.21.15.131:0/330422249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe990003fe0 0x7fe990004800 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe994007fd0 tx=0x7fe990005830 comp rx=0 tx=0).stop 2024-04-28T12:39:22.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.954+0000 7fe9b1a00640 1 -- 172.21.15.131:0/330422249 reap_dead start 2024-04-28T12:39:22.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.954+0000 7fe9b2400640 1 -- 172.21.15.131:0/330422249 shutdown_connections 2024-04-28T12:39:22.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.954+0000 7fe9b2400640 1 -- 172.21.15.131:0/330422249 >> 172.21.15.131:0/330422249 conn(0x7fe9ac0d4c50 msgr2=0x7fe9ac0d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:22.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.954+0000 7fe9b2400640 1 -- 172.21.15.131:0/330422249 shutdown_connections 2024-04-28T12:39:22.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.954+0000 7fe9b2400640 1 -- 172.21.15.131:0/330422249 wait complete. 2024-04-28T12:39:22.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.955+0000 7fe9b2400640 1 Processor -- start 2024-04-28T12:39:22.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.956+0000 7fe9b2400640 1 -- start start 2024-04-28T12:39:22.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe990003fe0 0x7fe9ac15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b2400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ac15e530 0x7fe9ac15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b2400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ac0cd080 0x7fe9ac0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b2400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac0d5770 con 0x7fe9ac0cd080 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b2400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac0d2db0 con 0x7fe9ac15e530 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac0d58f0 con 0x7fe990003fe0 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ac0cd080 0x7fe9ac0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ac0cd080 0x7fe9ac0cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56888/0 (socket says 172.21.15.131:56888) 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b1a00640 1 -- 172.21.15.131:0/1297926129 learned_addr learned my addr 172.21.15.131:0/1297926129 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9a3e00640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ac15e530 0x7fe9ac15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b1000640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe990003fe0 0x7fe9ac15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b1a00640 1 -- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe990003fe0 msgr2=0x7fe9ac15b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b1a00640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe990003fe0 0x7fe9ac15b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b1a00640 1 -- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ac15e530 msgr2=0x7fe9ac15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b1a00640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ac15e530 0x7fe9ac15ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b1a00640 1 -- 172.21.15.131:0/1297926129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac0cdde0 con 0x7fe9ac0cd080 2024-04-28T12:39:22.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.957+0000 7fe9b1000640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe990003fe0 0x7fe9ac15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:39:22.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.958+0000 7fe9b1a00640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ac0cd080 0x7fe9ac0cd500 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0b5c30 tx=0x7fe9a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:22.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.958+0000 7fe9a9600640 1 -- 172.21.15.131:0/1297926129 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4019070 con 0x7fe9ac0cd080 2024-04-28T12:39:22.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.958+0000 7fe9a9600640 1 -- 172.21.15.131:0/1297926129 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a4007cb0 con 0x7fe9ac0cd080 2024-04-28T12:39:22.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.958+0000 7fe9a9600640 1 -- 172.21.15.131:0/1297926129 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4007e60 con 0x7fe9ac0cd080 2024-04-28T12:39:22.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.958+0000 7fe9b2400640 1 -- 172.21.15.131:0/1297926129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac0ce0c0 con 0x7fe9ac0cd080 2024-04-28T12:39:22.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.958+0000 7fe9b2400640 1 -- 172.21.15.131:0/1297926129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac0ce680 con 0x7fe9ac0cd080 2024-04-28T12:39:22.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.959+0000 7fe9b2400640 1 -- 172.21.15.131:0/1297926129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ac069350 con 0x7fe9ac0cd080 2024-04-28T12:39:22.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.960+0000 7fe9a9600640 1 -- 172.21.15.131:0/1297926129 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9a401c070 con 0x7fe9ac0cd080 2024-04-28T12:39:22.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.961+0000 7fe9a9600640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe98806c920 0x7fe98806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:22.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.961+0000 7fe9a9600640 1 -- 172.21.15.131:0/1297926129 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe9a408b840 con 0x7fe9ac0cd080 2024-04-28T12:39:22.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.962+0000 7fe9b1000640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe98806c920 0x7fe98806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:22.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.963+0000 7fe9b1000640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe98806c920 0x7fe98806ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe99c004770 tx=0x7fe99c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:22.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:22.966+0000 7fe9a9600640 1 -- 172.21.15.131:0/1297926129 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe9a4059ad0 con 0x7fe9ac0cd080 2024-04-28T12:39:23.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.208+0000 7fe9b2400640 1 -- 172.21.15.131:0/1297926129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe9ac08ed80 con 0x7fe9ac0cd080 2024-04-28T12:39:23.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.208+0000 7fe9a9600640 1 -- 172.21.15.131:0/1297926129 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fe9a40593d0 con 0x7fe9ac0cd080 2024-04-28T12:39:23.267 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-28T12:39:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.211+0000 7fe9b2400640 1 -- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe98806c920 msgr2=0x7fe98806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.211+0000 7fe9b2400640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe98806c920 0x7fe98806ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe99c004770 tx=0x7fe99c008040 comp rx=0 tx=0).stop 2024-04-28T12:39:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.211+0000 7fe9b2400640 1 -- 172.21.15.131:0/1297926129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ac0cd080 msgr2=0x7fe9ac0cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.211+0000 7fe9b2400640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ac0cd080 0x7fe9ac0cd500 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0b5c30 tx=0x7fe9a4009da0 comp rx=0 tx=0).stop 2024-04-28T12:39:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.211+0000 7fe9b2400640 1 -- 172.21.15.131:0/1297926129 shutdown_connections 2024-04-28T12:39:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.211+0000 7fe9b2400640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe98806c920 0x7fe98806ede0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:23.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.211+0000 7fe9b2400640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9ac0cd080 0x7fe9ac0cd500 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.211+0000 7fe9b2400640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9ac15e530 0x7fe9ac15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.211+0000 7fe9b2400640 1 --2- 172.21.15.131:0/1297926129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe990003fe0 0x7fe9ac15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.211+0000 7fe9b2400640 1 -- 172.21.15.131:0/1297926129 >> 172.21.15.131:0/1297926129 conn(0x7fe9ac0d4c50 msgr2=0x7fe9ac0cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.212+0000 7fe9b2400640 1 -- 172.21.15.131:0/1297926129 shutdown_connections 2024-04-28T12:39:23.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23.212+0000 7fe9b2400640 1 -- 172.21.15.131:0/1297926129 wait complete. 2024-04-28T12:39:23.270 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-28T12:39:23.270 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T12:39:23.270 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-04-28T12:39:23.270 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-28T12:39:23.270 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-28T12:39:23.270 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-28T12:39:23.270 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-28T12:39:23.271 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T12:39:23.271 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-28T12:39:23.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:23.271 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-04-28T12:39:23.326 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T12:39:23.326 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T12:39:23.326 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T12:39:23.327 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:39:23.327 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-04-28T12:39:23.327 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:39:23.327 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:39:23.327 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:39:23.327 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-04-28T12:39:23.328 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:39:23.328 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:39:23.328 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-04-28T12:39:23.328 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:39:23.328 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:39:23.328 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T12:39:23.328 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-04-28T12:39:23.329 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T12:39:23.329 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42412sec preferred_lft 42412sec 2024-04-28T12:39:23.329 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-04-28T12:39:23.329 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:39:23.329 INFO:teuthology.orchestra.run.smithi131.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-28T12:39:23.329 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0a:69:73:69:7d:5e brd ff:ff:ff:ff:ff:ff 2024-04-28T12:39:23.330 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-28T12:39:23.330 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:39:23.330 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::682c:c6ff:fe45:6ca5/64 scope link 2024-04-28T12:39:23.330 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:39:23.330 INFO:teuthology.orchestra.run.smithi131.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T12:39:23.330 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 7e:da:99:10:fa:2a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:39:23.330 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::7cda:99ff:fe10:fa2a/64 scope link 2024-04-28T12:39:23.331 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:39:23.331 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-28T12:39:23.331 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0a:69:73:69:7d:5e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T12:39:23.331 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::869:73ff:fe69:7d5e/64 scope link 2024-04-28T12:39:23.331 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:39:23.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:23.332 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T12:39:23.411 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-04-28T12:39:23.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:23.415 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-04-28T12:39:23.468 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-04-28T12:39:23.469 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T12:39:23.469 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T12:39:23.469 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T12:39:23.469 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T12:39:23.469 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:39:23.469 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-04-28T12:39:23.469 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T12:39:23.469 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T12:39:23.469 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T12:39:23.470 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-04-28T12:39:23.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:39:23.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:39:23.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:23.682 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-04-28T12:39:23.728 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-28T12:39:23.728 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-28T12:39:23.729 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-28T12:39:23.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:23.729 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-04-28T12:39:23.806 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-28T12:39:23.807 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:39:23.807 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-28T12:39:23.862 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-28T12:39:23.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:39:23.863 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-28T12:39:23.912 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-04-28T12:39:23.993 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:39:24.048 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-04-28T12:39:24.048 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-04-28T12:39:24.048 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/selinux 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-04-28T12:39:24.049 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-04-28T12:39:24.050 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:39:24.051 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:39:24.051 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T12:39:24.051 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T12:39:24.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:24.052 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-04-28T12:39:24.101 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-28T12:39:24.101 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-28T12:39:24.106 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-04-28T12:39:24.172 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:39:24.215 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-04-28T12:39:24.215 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-04-28T12:39:24.215 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-04-28T12:39:24.215 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-04-28T12:39:24.215 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-04-28T12:39:24.215 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-04-28T12:39:24.215 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-04-28T12:39:24.215 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/selinux 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:39:24.216 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:39:24.217 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-04-28T12:39:24.217 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-04-28T12:39:24.217 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-04-28T12:39:24.217 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-04-28T12:39:24.217 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-04-28T12:39:24.217 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-04-28T12:39:24.217 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-04-28T12:39:24.217 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:39:24.217 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:39:24.217 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T12:39:24.217 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T12:39:24.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:24.218 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-04-28T12:39:24.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-04-28T12:39:24.271+0000 7f451c785480 -1 init, newargv = 0x562f44686460 newargc=15 2024-04-28T12:39:24.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[49497]: starting ceph client 2024-04-28T12:39:24.278 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[49497]: starting fuse 2024-04-28T12:39:24.300 INFO:teuthology.orchestra.run.smithi131.stdout:41 2024-04-28T12:39:24.301 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-28T12:39:24.301 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-04-28T12:39:24.301 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-04-28T12:39:24.301 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-04-28T12:39:24.301 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-04-28T12:39:24.301 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-04-28T12:39:24.301 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:39:24.301 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-04-28T12:39:24.301 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-28T12:39:24.301 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-04-28T12:39:24.301 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-28T12:39:24.301 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-04-28T12:39:24.302 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-04-28T12:39:24.482 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.49497.asok 2024-04-28T12:39:24.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:39:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:39:24.498 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49497.asok 2024-04-28T12:39:24.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:24.502 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49497.asok status 2024-04-28T12:39:24.725 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:39:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-04-28T12:39:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-28T12:39:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-28T12:39:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-04-28T12:39:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-04-28T12:39:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-28T12:39:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "49497", 2024-04-28T12:39:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-04-28T12:39:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-04-28T12:39:24.726 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5709, 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 5709 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3006728348 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-04-28T12:39:24.727 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-04-28T12:39:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3006728348 2024-04-28T12:39:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-04-28T12:39:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.5709 192.168.0.1:0/3006728348", 2024-04-28T12:39:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/3006728348", 2024-04-28T12:39:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-04-28T12:39:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 17, 2024-04-28T12:39:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 38, 2024-04-28T12:39:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-04-28T12:39:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-04-28T12:39:24.728 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-04-28T12:39:24.728 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:39:24.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:24.740 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-04-28T12:39:24.740 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-28T12:39:24.794 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-04-28T12:39:24.794 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-28T12:39:24.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:24.794 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-28T12:39:24.879 DEBUG:teuthology.orchestra.run.smithi131:> mktemp -d --suffix=xfstests-dev 2024-04-28T12:39:24.932 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.IDf8sLWWdqxfstests-dev 2024-04-28T12:39:24.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:24.932 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.IDf8sLWWdqxfstests-dev) 2024-04-28T12:39:25.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:25.020 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.IDf8sLWWdqxfstests-dev) 2024-04-28T12:39:25.090 INFO:teuthology.orchestra.run.smithi131.stderr:Cloning into '/tmp/tmp.IDf8sLWWdqxfstests-dev'... 2024-04-28T12:39:26.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:26.152 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-04-28T12:39:26.220 DEBUG:teuthology.orchestra.run.smithi131:> mktemp -d --suffix=test 2024-04-28T12:39:26.273 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.7Yn4nqdfDitest 2024-04-28T12:39:26.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:39:26.273 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-04-28T12:39:26.330 DEBUG:teuthology.orchestra.run.smithi131:> mktemp -d --suffix=scratch 2024-04-28T12:39:26.382 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.iLZuwKiioGscratch 2024-04-28T12:39:26.382 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-04-28T12:39:26.382 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-04-28T12:39:27.159 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:06:35 ago on Sun 28 Apr 2024 12:32:52 PM UTC. 2024-04-28T12:39:27.244 INFO:teuthology.orchestra.run.smithi131.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-04-28T12:39:27.245 INFO:teuthology.orchestra.run.smithi131.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-04-28T12:39:27.246 INFO:teuthology.orchestra.run.smithi131.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-04-28T12:39:27.247 INFO:teuthology.orchestra.run.smithi131.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-04-28T12:39:27.247 INFO:teuthology.orchestra.run.smithi131.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-04-28T12:39:27.249 INFO:teuthology.orchestra.run.smithi131.stdout:Package e2fsprogs-1.46.5-5.el9.x86_64 is already installed. 2024-04-28T12:39:27.251 INFO:teuthology.orchestra.run.smithi131.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-04-28T12:39:27.252 INFO:teuthology.orchestra.run.smithi131.stdout:Package gcc-11.4.1-3.el9.x86_64 is already installed. 2024-04-28T12:39:27.253 INFO:teuthology.orchestra.run.smithi131.stdout:Package git-2.43.0-1.el9.x86_64 is already installed. 2024-04-28T12:39:27.255 INFO:teuthology.orchestra.run.smithi131.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-28T12:39:27.256 INFO:teuthology.orchestra.run.smithi131.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-04-28T12:39:27.258 INFO:teuthology.orchestra.run.smithi131.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-04-28T12:39:27.259 INFO:teuthology.orchestra.run.smithi131.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-04-28T12:39:27.260 INFO:teuthology.orchestra.run.smithi131.stdout:Package lvm2-9:2.03.23-2.el9.x86_64 is already installed. 2024-04-28T12:39:27.261 INFO:teuthology.orchestra.run.smithi131.stdout:Package make-1:4.3-8.el9.x86_64 is already installed. 2024-04-28T12:39:27.261 INFO:teuthology.orchestra.run.smithi131.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-04-28T12:39:27.262 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-3.9.18-3.el9.x86_64 is already installed. 2024-04-28T12:39:27.263 INFO:teuthology.orchestra.run.smithi131.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-04-28T12:39:27.264 INFO:teuthology.orchestra.run.smithi131.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-04-28T12:39:27.266 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-28T12:39:27.313 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-04-28T12:39:27.314 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-04-28T12:39:27.314 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout: fio x86_64 3.35-1.el9 appstream 6.2 M 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout: gdbm-devel x86_64 1:1.19-4.el9 crb 59 k 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-devel x86_64 5.14.0-437.el9 appstream 16 M 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout: libcap-devel x86_64 2.48-9.el9 appstream 41 k 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout: udftools x86_64 2.2-5.el9 appstream 149 k 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout: gdbm x86_64 1:1.19-4.el9 crb 132 k 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout: libnbd x86_64 1.18.1-3.el9 appstream 166 k 2024-04-28T12:39:27.315 INFO:teuthology.orchestra.run.smithi131.stdout: rmt x86_64 2:1.6-6.el9 baseos 46 k 2024-04-28T12:39:27.316 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-04-28T12:39:27.316 INFO:teuthology.orchestra.run.smithi131.stdout: fio-engine-http x86_64 3.35-1.el9 appstream 18 k 2024-04-28T12:39:27.316 INFO:teuthology.orchestra.run.smithi131.stdout: fio-engine-libaio x86_64 3.35-1.el9 appstream 16 k 2024-04-28T12:39:27.316 INFO:teuthology.orchestra.run.smithi131.stdout: fio-engine-nbd x86_64 3.35-1.el9 appstream 15 k 2024-04-28T12:39:27.316 INFO:teuthology.orchestra.run.smithi131.stdout: fio-engine-rados x86_64 3.35-1.el9 appstream 16 k 2024-04-28T12:39:27.316 INFO:teuthology.orchestra.run.smithi131.stdout: fio-engine-rbd x86_64 3.35-1.el9 appstream 16 k 2024-04-28T12:39:27.316 INFO:teuthology.orchestra.run.smithi131.stdout: fio-engine-rdma x86_64 3.35-1.el9 appstream 21 k 2024-04-28T12:39:27.316 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:27.316 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-04-28T12:39:27.316 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-04-28T12:39:27.316 INFO:teuthology.orchestra.run.smithi131.stdout:Install 17 Packages 2024-04-28T12:39:27.316 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:39:27.317 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 24 M 2024-04-28T12:39:27.317 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 78 M 2024-04-28T12:39:27.317 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-04-28T12:39:28.100 INFO:teuthology.orchestra.run.smithi131.stdout:(1/17): fio-engine-http-3.35-1.el9.x86_64.rpm 134 kB/s | 18 kB 00:00 2024-04-28T12:39:28.124 INFO:teuthology.orchestra.run.smithi131.stdout:(2/17): rmt-1.6-6.el9.x86_64.rpm 290 kB/s | 46 kB 00:00 2024-04-28T12:39:28.149 INFO:teuthology.orchestra.run.smithi131.stdout:(3/17): fio-engine-libaio-3.35-1.el9.x86_64.rpm 325 kB/s | 16 kB 00:00 2024-04-28T12:39:28.206 INFO:teuthology.orchestra.run.smithi131.stdout:(4/17): fio-engine-nbd-3.35-1.el9.x86_64.rpm 179 kB/s | 15 kB 00:00 2024-04-28T12:39:28.242 INFO:teuthology.orchestra.run.smithi131.stdout:(5/17): fio-engine-rados-3.35-1.el9.x86_64.rpm 173 kB/s | 16 kB 00:00 2024-04-28T12:39:28.267 INFO:teuthology.orchestra.run.smithi131.stdout:(6/17): fio-engine-rbd-3.35-1.el9.x86_64.rpm 275 kB/s | 16 kB 00:00 2024-04-28T12:39:28.309 INFO:teuthology.orchestra.run.smithi131.stdout:(7/17): fio-engine-rdma-3.35-1.el9.x86_64.rpm 312 kB/s | 21 kB 00:00 2024-04-28T12:39:28.384 INFO:teuthology.orchestra.run.smithi131.stdout:(8/17): libcap-devel-2.48-9.el9.x86_64.rpm 548 kB/s | 41 kB 00:00 2024-04-28T12:39:28.493 INFO:teuthology.orchestra.run.smithi131.stdout:(9/17): libnbd-1.18.1-3.el9.x86_64.rpm 1.5 MB/s | 166 kB 00:00 2024-04-28T12:39:28.585 INFO:teuthology.orchestra.run.smithi131.stdout:(10/17): fio-3.35-1.el9.x86_64.rpm 10 MB/s | 6.2 MB 00:00 2024-04-28T12:39:28.635 INFO:teuthology.orchestra.run.smithi131.stdout:(11/17): udftools-2.2-5.el9.x86_64.rpm 3.0 MB/s | 149 kB 00:00 2024-04-28T12:39:28.752 INFO:teuthology.orchestra.run.smithi131.stdout:(12/17): sqlite-3.34.1-7.el9.x86_64.rpm 2.8 MB/s | 748 kB 00:00 2024-04-28T12:39:28.777 INFO:teuthology.orchestra.run.smithi131.stdout:(13/17): gdbm-1.19-4.el9.x86_64.rpm 939 kB/s | 132 kB 00:00 2024-04-28T12:39:28.986 INFO:teuthology.orchestra.run.smithi131.stdout:(14/17): kernel-devel-5.14.0-437.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-28T12:39:29.036 INFO:teuthology.orchestra.run.smithi131.stdout:(15/17): gdbm-devel-1.19-4.el9.x86_64.rpm 207 kB/s | 59 kB 00:00 2024-04-28T12:39:29.419 INFO:teuthology.orchestra.run.smithi131.stdout:(16/17): dump-0.4-0.51.b47.el9.x86_64.rpm 137 kB/s | 88 kB 00:00 2024-04-28T12:39:29.561 INFO:teuthology.orchestra.run.smithi131.stdout:(17/17): indent-2.2.13-5.el9.x86_64.rpm 327 kB/s | 179 kB 00:00 2024-04-28T12:39:29.565 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-04-28T12:39:29.566 INFO:teuthology.orchestra.run.smithi131.stdout:Total 11 MB/s | 24 MB 00:02 2024-04-28T12:39:30.296 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-04-28T12:39:30.407 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-04-28T12:39:30.407 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-04-28T12:39:31.283 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-04-28T12:39:31.283 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-04-28T12:39:32.504 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-04-28T12:39:32.584 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gdbm-1:1.19-4.el9.x86_64 1/17 2024-04-28T12:39:32.659 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 2/17 2024-04-28T12:39:32.693 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 3/17 2024-04-28T12:39:32.727 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 4/17 2024-04-28T12:39:32.762 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 5/17 2024-04-28T12:39:32.796 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 6/17 2024-04-28T12:39:32.829 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 7/17 2024-04-28T12:39:33.090 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 8/17 2024-04-28T12:39:33.150 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fio-3.35-1.el9.x86_64 9/17 2024-04-28T12:39:33.193 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rmt-2:1.6-6.el9.x86_64 10/17 2024-04-28T12:39:33.236 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 11/17 2024-04-28T12:39:33.323 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gdbm-devel-1:1.19-4.el9.x86_64 12/17 2024-04-28T12:39:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : indent-2.2.13-5.el9.x86_64 13/17 2024-04-28T12:39:33.460 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : udftools-2.2-5.el9.x86_64 14/17 2024-04-28T12:39:33.556 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 15/17 2024-04-28T12:39:45.016 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 16/17 2024-04-28T12:39:45.416 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-devel-5.14.0-437.el9.x86_64 17/17 2024-04-28T12:39:45.900 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:39:45.906 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:40:16.019 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:40:16.027 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:40:35.905 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-devel-5.14.0-437.el9.x86_64 17/17 2024-04-28T12:40:35.905 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 1/17 2024-04-28T12:40:35.905 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fio-3.35-1.el9.x86_64 2/17 2024-04-28T12:40:35.905 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 3/17 2024-04-28T12:40:35.905 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 4/17 2024-04-28T12:40:35.906 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 5/17 2024-04-28T12:40:35.906 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 6/17 2024-04-28T12:40:35.906 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 7/17 2024-04-28T12:40:35.906 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 8/17 2024-04-28T12:40:35.906 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-devel-5.14.0-437.el9.x86_64 9/17 2024-04-28T12:40:35.906 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 10/17 2024-04-28T12:40:35.906 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 11/17 2024-04-28T12:40:35.906 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 12/17 2024-04-28T12:40:35.906 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : udftools-2.2-5.el9.x86_64 13/17 2024-04-28T12:40:35.906 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gdbm-1:1.19-4.el9.x86_64 14/17 2024-04-28T12:40:35.906 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gdbm-devel-1:1.19-4.el9.x86_64 15/17 2024-04-28T12:40:35.906 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/17 2024-04-28T12:40:36.451 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : indent-2.2.13-5.el9.x86_64 17/17 2024-04-28T12:40:36.451 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:40:36.451 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-04-28T12:40:36.451 INFO:teuthology.orchestra.run.smithi131.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-04-28T12:40:36.451 INFO:teuthology.orchestra.run.smithi131.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-04-28T12:40:36.451 INFO:teuthology.orchestra.run.smithi131.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-04-28T12:40:36.451 INFO:teuthology.orchestra.run.smithi131.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-04-28T12:40:36.451 INFO:teuthology.orchestra.run.smithi131.stdout: gdbm-1:1.19-4.el9.x86_64 gdbm-devel-1:1.19-4.el9.x86_64 2024-04-28T12:40:36.451 INFO:teuthology.orchestra.run.smithi131.stdout: indent-2.2.13-5.el9.x86_64 kernel-devel-5.14.0-437.el9.x86_64 2024-04-28T12:40:36.451 INFO:teuthology.orchestra.run.smithi131.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2024-04-28T12:40:36.451 INFO:teuthology.orchestra.run.smithi131.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-04-28T12:40:36.452 INFO:teuthology.orchestra.run.smithi131.stdout: udftools-2.2-5.el9.x86_64 2024-04-28T12:40:36.452 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:40:36.452 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-04-28T12:40:36.942 DEBUG:teuthology.orchestra.run.smithi131:> sudo useradd -m fsgqa 2024-04-28T12:40:37.013 INFO:teuthology.orchestra.run.smithi131.stderr:useradd: user 'fsgqa' already exists 2024-04-28T12:40:37.018 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-28T12:40:37.018 DEBUG:teuthology.orchestra.run.smithi131:> sudo groupadd fsgqa 2024-04-28T12:40:37.078 INFO:teuthology.orchestra.run.smithi131.stderr:groupadd: group 'fsgqa' already exists 2024-04-28T12:40:37.081 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-28T12:40:37.082 DEBUG:teuthology.orchestra.run.smithi131:> sudo useradd fsgqa2 2024-04-28T12:40:37.750 DEBUG:teuthology.orchestra.run.smithi131:> sudo useradd 123456-fsgqa 2024-04-28T12:40:38.391 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-04-28T12:40:38.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.574+0000 7fd19be00640 1 Processor -- start 2024-04-28T12:40:38.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.574+0000 7fd19be00640 1 -- start start 2024-04-28T12:40:38.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.576+0000 7fd19be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd19c1463e0 0x7fd19c0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:38.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.576+0000 7fd19be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd19c161460 0x7fd19c160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:38.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.576+0000 7fd19be00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd19c0b50e0 con 0x7fd19c160d90 2024-04-28T12:40:38.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.576+0000 7fd19be00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd19c1611e0 con 0x7fd19c1463e0 2024-04-28T12:40:38.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.576+0000 7fd19be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd19c0b4560 con 0x7fd19c161460 2024-04-28T12:40:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.577+0000 7fd19aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd19c1463e0 0x7fd19c0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.577+0000 7fd19aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd19c1463e0 0x7fd19c0b58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47142/0 (socket says 172.21.15.131:47142) 2024-04-28T12:40:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.577+0000 7fd19aa00640 1 -- 172.21.15.131:0/3354717870 learned_addr learned my addr 172.21.15.131:0/3354717870 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:40:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.577+0000 7fd192000640 1 --2- 172.21.15.131:0/3354717870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd19c161460 0x7fd19c160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:38.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.577+0000 7fd19aa00640 1 -- 172.21.15.131:0/3354717870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd19c161460 msgr2=0x7fd19c160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:40:38.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.577+0000 7fd19aa00640 1 --2- 172.21.15.131:0/3354717870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd19c161460 0x7fd19c160850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:38.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.577+0000 7fd19aa00640 1 -- 172.21.15.131:0/3354717870 >> v1:172.21.15.157:6789/0 conn(0x7fd19c160d90 legacy=0x7fd19c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:40:38.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.577+0000 7fd19aa00640 1 -- 172.21.15.131:0/3354717870 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c0b4840 con 0x7fd19c1463e0 2024-04-28T12:40:38.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.579+0000 7fd19aa00640 1 --2- 172.21.15.131:0/3354717870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd19c1463e0 0x7fd19c0b58b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009870 tx=0x7fd18c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2daeeba9b7288da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:38.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.579+0000 7fd19a000640 1 -- 172.21.15.131:0/3354717870 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c037070 con 0x7fd19c1463e0 2024-04-28T12:40:38.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.579+0000 7fd19a000640 1 -- 172.21.15.131:0/3354717870 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd18c032440 con 0x7fd19c1463e0 2024-04-28T12:40:38.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.579+0000 7fd19a000640 1 -- 172.21.15.131:0/3354717870 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c032810 con 0x7fd19c1463e0 2024-04-28T12:40:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.580+0000 7fd19be00640 1 -- 172.21.15.131:0/3354717870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd19c1463e0 msgr2=0x7fd19c0b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.580+0000 7fd19be00640 1 --2- 172.21.15.131:0/3354717870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd19c1463e0 0x7fd19c0b58b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009870 tx=0x7fd18c02a180 comp rx=0 tx=0).stop 2024-04-28T12:40:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.580+0000 7fd19be00640 1 -- 172.21.15.131:0/3354717870 shutdown_connections 2024-04-28T12:40:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.580+0000 7fd19be00640 1 --2- 172.21.15.131:0/3354717870 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd19c161460 0x7fd19c160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.580+0000 7fd19be00640 1 --2- 172.21.15.131:0/3354717870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd19c1463e0 0x7fd19c0b58b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.580+0000 7fd19be00640 1 -- 172.21.15.131:0/3354717870 >> 172.21.15.131:0/3354717870 conn(0x7fd19c0b26b0 msgr2=0x7fd19c0b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.580+0000 7fd19be00640 1 -- 172.21.15.131:0/3354717870 shutdown_connections 2024-04-28T12:40:38.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.580+0000 7fd19be00640 1 -- 172.21.15.131:0/3354717870 wait complete. 2024-04-28T12:40:38.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.582+0000 7fd19be00640 1 Processor -- start 2024-04-28T12:40:38.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.582+0000 7fd19be00640 1 -- start start 2024-04-28T12:40:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.584+0000 7fd19be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd19c1463e0 0x7fd19c1e72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.584+0000 7fd19be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd19c160d90 0x7fd19c1e7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.584+0000 7fd19be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd19c161460 0x7fd19c13bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.584+0000 7fd19be00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1467c0 con 0x7fd19c160d90 2024-04-28T12:40:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.584+0000 7fd19be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c0b4e00 con 0x7fd19c161460 2024-04-28T12:40:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.584+0000 7fd19be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd19c0b5df0 con 0x7fd19c1463e0 2024-04-28T12:40:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.584+0000 7fd192000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd19c160d90 0x7fd19c1e7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:38.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.584+0000 7fd192000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd19c160d90 0x7fd19c1e7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47154/0 (socket says 172.21.15.131:47154) 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.584+0000 7fd192000640 1 -- 172.21.15.131:0/2125886442 learned_addr learned my addr 172.21.15.131:0/2125886442 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.585+0000 7fd19aa00640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd19c1463e0 0x7fd19c1e72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.585+0000 7fd19b400640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd19c161460 0x7fd19c13bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.585+0000 7fd192000640 1 -- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd19c1463e0 msgr2=0x7fd19c1e72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.585+0000 7fd192000640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd19c1463e0 0x7fd19c1e72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.585+0000 7fd192000640 1 -- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd19c161460 msgr2=0x7fd19c13bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.585+0000 7fd192000640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd19c161460 0x7fd19c13bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.585+0000 7fd192000640 1 -- 172.21.15.131:0/2125886442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c145750 con 0x7fd19c160d90 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.585+0000 7fd19b400640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd19c161460 0x7fd19c13bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.585+0000 7fd192000640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd19c160d90 0x7fd19c1e7cf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c7f0 tx=0x7fd18400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.585+0000 7fd193400640 1 -- 172.21.15.131:0/2125886442 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184015070 con 0x7fd19c160d90 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.585+0000 7fd193400640 1 -- 172.21.15.131:0/2125886442 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd184004240 con 0x7fd19c160d90 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.585+0000 7fd19be00640 1 -- 172.21.15.131:0/2125886442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c1e67c0 con 0x7fd19c160d90 2024-04-28T12:40:38.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.586+0000 7fd193400640 1 -- 172.21.15.131:0/2125886442 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184007620 con 0x7fd19c160d90 2024-04-28T12:40:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.586+0000 7fd19be00640 1 -- 172.21.15.131:0/2125886442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c1e77e0 con 0x7fd19c160d90 2024-04-28T12:40:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.587+0000 7fd19be00640 1 -- 172.21.15.131:0/2125886442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19c069350 con 0x7fd19c160d90 2024-04-28T12:40:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.588+0000 7fd193400640 1 -- 172.21.15.131:0/2125886442 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1840079d0 con 0x7fd19c160d90 2024-04-28T12:40:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.589+0000 7fd193400640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd17806c8f0 0x7fd17806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:38.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.589+0000 7fd193400640 1 -- 172.21.15.131:0/2125886442 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fd18408c1a0 con 0x7fd19c160d90 2024-04-28T12:40:38.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.589+0000 7fd19aa00640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd17806c8f0 0x7fd17806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:38.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.591+0000 7fd19aa00640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd17806c8f0 0x7fd17806edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd18c034000 tx=0x7fd18c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:38.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.594+0000 7fd193400640 1 -- 172.21.15.131:0/2125886442 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd18405a410 con 0x7fd19c160d90 2024-04-28T12:40:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.873+0000 7fd19be00640 1 -- 172.21.15.131:0/2125886442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd19c073f40 con 0x7fd19c160d90 2024-04-28T12:40:38.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.873+0000 7fd193400640 1 -- 172.21.15.131:0/2125886442 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7fd184059d10 con 0x7fd19c160d90 2024-04-28T12:40:38.870 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:40:38.871 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":1,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","modified":"2024-04-28T12:36:17.572247Z","created":"2024-04-28T12:36:17.572247Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3301","nonce":0},{"type":"v1","addr":"172.21.15.157:6790","nonce":0}]},"addr":"172.21.15.157:6790/0","public_addr":"172.21.15.157:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-28T12:40:38.871 INFO:teuthology.orchestra.run.smithi131.stderr:dumped monmap epoch 1 2024-04-28T12:40:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.876+0000 7fd19be00640 1 -- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd17806c8f0 msgr2=0x7fd17806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.876+0000 7fd19be00640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd17806c8f0 0x7fd17806edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd18c034000 tx=0x7fd18c02e000 comp rx=0 tx=0).stop 2024-04-28T12:40:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.876+0000 7fd19be00640 1 -- 172.21.15.131:0/2125886442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd19c160d90 msgr2=0x7fd19c1e7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.876+0000 7fd19be00640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd19c160d90 0x7fd19c1e7cf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c7f0 tx=0x7fd18400ccb0 comp rx=0 tx=0).stop 2024-04-28T12:40:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.876+0000 7fd19be00640 1 -- 172.21.15.131:0/2125886442 shutdown_connections 2024-04-28T12:40:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.876+0000 7fd19be00640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fd17806c8f0 0x7fd17806edb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.876+0000 7fd19be00640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd19c161460 0x7fd19c13bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.876+0000 7fd19be00640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd19c160d90 0x7fd19c1e7cf0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.876+0000 7fd19be00640 1 --2- 172.21.15.131:0/2125886442 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd19c1463e0 0x7fd19c1e72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.876+0000 7fd19be00640 1 -- 172.21.15.131:0/2125886442 >> 172.21.15.131:0/2125886442 conn(0x7fd19c0b26b0 msgr2=0x7fd19c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.877+0000 7fd19be00640 1 -- 172.21.15.131:0/2125886442 shutdown_connections 2024-04-28T12:40:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:38.877+0000 7fd19be00640 1 -- 172.21.15.131:0/2125886442 wait complete. 2024-04-28T12:40:38.895 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2024-04-28T12:40:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.071+0000 7ffaee800640 1 Processor -- start 2024-04-28T12:40:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.072+0000 7ffaee800640 1 -- start start 2024-04-28T12:40:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.073+0000 7ffaee800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffae8145330 0x7ffae80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.073+0000 7ffaee800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffae81628e0 0x7ffae8162d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.073+0000 7ffaee800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae80af720 0x7ffae80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.073+0000 7ffaee800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ffae8145710 con 0x7ffae81628e0 2024-04-28T12:40:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.073+0000 7ffaee800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ffae8073dc0 con 0x7ffae8145330 2024-04-28T12:40:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.073+0000 7ffaee800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffae80ae960 con 0x7ffae80af720 2024-04-28T12:40:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.073+0000 7ffaeca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffae81628e0 0x7ffae8162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.073+0000 7ffaeca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffae81628e0 0x7ffae8162d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47156/0 (socket says 172.21.15.131:47156) 2024-04-28T12:40:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.073+0000 7ffaeca00640 1 -- 172.21.15.131:0/184813945 learned_addr learned my addr 172.21.15.131:0/184813945 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:40:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.074+0000 7ffaeca00640 1 -- 172.21.15.131:0/184813945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae80af720 msgr2=0x7ffae80afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:40:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.074+0000 7ffaed400640 1 --2- 172.21.15.131:0/184813945 >> v2:172.21.15.157:3300/0 conn(0x7ffae8145330 0x7ffae80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.074+0000 7ffaede00640 1 --2- 172.21.15.131:0/184813945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae80af720 0x7ffae80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.074+0000 7ffaeca00640 1 --2- 172.21.15.131:0/184813945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae80af720 0x7ffae80afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.074+0000 7ffaeca00640 1 -- 172.21.15.131:0/184813945 >> v2:172.21.15.157:3300/0 conn(0x7ffae8145330 msgr2=0x7ffae80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:40:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.074+0000 7ffaeca00640 1 --2- 172.21.15.131:0/184813945 >> v2:172.21.15.157:3300/0 conn(0x7ffae8145330 0x7ffae80b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.074+0000 7ffaeca00640 1 -- 172.21.15.131:0/184813945 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae80aec40 con 0x7ffae81628e0 2024-04-28T12:40:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.075+0000 7ffaeca00640 1 --2- 172.21.15.131:0/184813945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae81628e0 0x7ffae8162d70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ffae80d07f0 tx=0x7ffad402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405319fc1e52bf6c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.075+0000 7ffae7e00640 1 -- 172.21.15.131:0/184813945 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4037070 con 0x7ffae81628e0 2024-04-28T12:40:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.075+0000 7ffae7e00640 1 -- 172.21.15.131:0/184813945 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad4032440 con 0x7ffae81628e0 2024-04-28T12:40:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.075+0000 7ffae7e00640 1 -- 172.21.15.131:0/184813945 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4032810 con 0x7ffae81628e0 2024-04-28T12:40:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.076+0000 7ffaee800640 1 -- 172.21.15.131:0/184813945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae81628e0 msgr2=0x7ffae8162d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.076+0000 7ffaee800640 1 --2- 172.21.15.131:0/184813945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae81628e0 0x7ffae8162d70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ffae80d07f0 tx=0x7ffad402a1d0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.076+0000 7ffaee800640 1 -- 172.21.15.131:0/184813945 shutdown_connections 2024-04-28T12:40:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.076+0000 7ffaee800640 1 --2- 172.21.15.131:0/184813945 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae81628e0 0x7ffae8162d70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.076+0000 7ffaee800640 1 --2- 172.21.15.131:0/184813945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae80af720 0x7ffae80afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.076+0000 7ffaee800640 1 --2- 172.21.15.131:0/184813945 >> v2:172.21.15.157:3300/0 conn(0x7ffae8145330 0x7ffae80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.076+0000 7ffaee800640 1 -- 172.21.15.131:0/184813945 >> 172.21.15.131:0/184813945 conn(0x7ffae8068090 msgr2=0x7ffae8068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.076+0000 7ffaee800640 1 -- 172.21.15.131:0/184813945 shutdown_connections 2024-04-28T12:40:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.076+0000 7ffaee800640 1 -- 172.21.15.131:0/184813945 wait complete. 2024-04-28T12:40:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.078+0000 7ffaee800640 1 Processor -- start 2024-04-28T12:40:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.078+0000 7ffaee800640 1 -- start start 2024-04-28T12:40:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.079+0000 7ffaee800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae80af720 0x7ffae8161710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.079+0000 7ffaee800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae8145330 0x7ffae80b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.079+0000 7ffaee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae81628e0 0x7ffae80b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.079+0000 7ffaee800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffae80b3e80 con 0x7ffae81628e0 2024-04-28T12:40:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.079+0000 7ffaee800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffae80af200 con 0x7ffae80af720 2024-04-28T12:40:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.079+0000 7ffaee800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffae80b00d0 con 0x7ffae8145330 2024-04-28T12:40:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.079+0000 7ffaede00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae81628e0 0x7ffae80b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.079+0000 7ffaede00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae81628e0 0x7ffae80b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47164/0 (socket says 172.21.15.131:47164) 2024-04-28T12:40:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.079+0000 7ffaede00640 1 -- 172.21.15.131:0/2290432047 learned_addr learned my addr 172.21.15.131:0/2290432047 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:40:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.080+0000 7ffaeca00640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae8145330 0x7ffae80b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.080+0000 7ffaed400640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae80af720 0x7ffae8161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.080+0000 7ffaede00640 1 -- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae8145330 msgr2=0x7ffae80b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.080+0000 7ffaede00640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae8145330 0x7ffae80b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.080+0000 7ffaede00640 1 -- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae80af720 msgr2=0x7ffae8161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.080+0000 7ffaede00640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae80af720 0x7ffae8161710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.080+0000 7ffaede00640 1 -- 172.21.15.131:0/2290432047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae81f7e70 con 0x7ffae81628e0 2024-04-28T12:40:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.080+0000 7ffaede00640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae81628e0 0x7ffae80b30b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ffae000c4c0 tx=0x7ffae000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.080+0000 7ffae5600640 1 -- 172.21.15.131:0/2290432047 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0015070 con 0x7ffae81628e0 2024-04-28T12:40:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.080+0000 7ffae5600640 1 -- 172.21.15.131:0/2290432047 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae0004220 con 0x7ffae81628e0 2024-04-28T12:40:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.080+0000 7ffae5600640 1 -- 172.21.15.131:0/2290432047 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0003610 con 0x7ffae81628e0 2024-04-28T12:40:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.080+0000 7ffaee800640 1 -- 172.21.15.131:0/2290432047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae81f80e0 con 0x7ffae81628e0 2024-04-28T12:40:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.081+0000 7ffaee800640 1 -- 172.21.15.131:0/2290432047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae81f8980 con 0x7ffae81628e0 2024-04-28T12:40:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.082+0000 7ffaee800640 1 -- 172.21.15.131:0/2290432047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae81f8b80 con 0x7ffae81628e0 2024-04-28T12:40:39.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.083+0000 7ffae5600640 1 -- 172.21.15.131:0/2290432047 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffae00037b0 con 0x7ffae81628e0 2024-04-28T12:40:39.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.084+0000 7ffae5600640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffac806c8f0 0x7ffac806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:39.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.084+0000 7ffae5600640 1 -- 172.21.15.131:0/2290432047 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7ffae008be60 con 0x7ffae81628e0 2024-04-28T12:40:39.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.084+0000 7ffaed400640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffac806c8f0 0x7ffac806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:39.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.086+0000 7ffaed400640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffac806c8f0 0x7ffac806edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffae81f5950 tx=0x7ffadc0083c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:39.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.088+0000 7ffae5600640 1 -- 172.21.15.131:0/2290432047 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffae005a040 con 0x7ffae81628e0 2024-04-28T12:40:39.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.408+0000 7ffaee800640 1 -- 172.21.15.131:0/2290432047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) -- 0x7ffae8069350 con 0x7ffae81628e0 2024-04-28T12:40:39.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.408+0000 7ffae5600640 1 -- 172.21.15.131:0/2290432047 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v12) ==== 82+0+151 (secure 0 0 0) 0x7ffae0059940 con 0x7ffae81628e0 2024-04-28T12:40:39.405 INFO:teuthology.orchestra.run.smithi131.stdout:[client.admin] 2024-04-28T12:40:39.405 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDBQi5mTaCENhAAE+Zs1n7PlmCjxUIQnxjpiQ== 2024-04-28T12:40:39.405 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow *" 2024-04-28T12:40:39.405 INFO:teuthology.orchestra.run.smithi131.stdout: caps mgr = "allow *" 2024-04-28T12:40:39.405 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow *" 2024-04-28T12:40:39.405 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow *" 2024-04-28T12:40:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.411+0000 7ffaee800640 1 -- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffac806c8f0 msgr2=0x7ffac806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.411+0000 7ffaee800640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffac806c8f0 0x7ffac806edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffae81f5950 tx=0x7ffadc0083c0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.411+0000 7ffaee800640 1 -- 172.21.15.131:0/2290432047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae81628e0 msgr2=0x7ffae80b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.411+0000 7ffaee800640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae81628e0 0x7ffae80b30b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ffae000c4c0 tx=0x7ffae000c980 comp rx=0 tx=0).stop 2024-04-28T12:40:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.411+0000 7ffaee800640 1 -- 172.21.15.131:0/2290432047 shutdown_connections 2024-04-28T12:40:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.411+0000 7ffaee800640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffac806c8f0 0x7ffac806edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.411+0000 7ffaee800640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffae81628e0 0x7ffae80b30b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.411+0000 7ffaee800640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffae8145330 0x7ffae80b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.411+0000 7ffaee800640 1 --2- 172.21.15.131:0/2290432047 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffae80af720 0x7ffae8161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.411+0000 7ffaee800640 1 -- 172.21.15.131:0/2290432047 >> 172.21.15.131:0/2290432047 conn(0x7ffae8068090 msgr2=0x7ffae815e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.411+0000 7ffaee800640 1 -- 172.21.15.131:0/2290432047 shutdown_connections 2024-04-28T12:40:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:40:39.411+0000 7ffaee800640 1 -- 172.21.15.131:0/2290432047 wait complete. 2024-04-28T12:40:39.426 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-04-28T12:40:39.467 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.hqIhKsgWIt 2024-04-28T12:40:39.467 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:40:39.467 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/tmp/tmp.hqIhKsgWIt 2024-04-28T12:40:39.547 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:40:39.547 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/tmp/tmp.IDf8sLWWdqxfstests-dev/local.config 2024-04-28T12:40:39.648 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.7Yn4nqdfDitest export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.iLZuwKiioGscratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.131:6789 -k /tmp/tmp.hqIhKsgWIt --client_mountpoint /test" 2024-04-28T12:40:39.648 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:40:39.648 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/tmp/tmp.IDf8sLWWdqxfstests-dev/ceph.exclude 2024-04-28T12:40:39.752 DEBUG:teuthology.orchestra.run.smithi131:> (cd /tmp/tmp.IDf8sLWWdqxfstests-dev && exec sudo make) 2024-04-28T12:40:39.869 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize -cfi 2024-04-28T12:40:40.579 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-28T12:40:40.579 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize: copying file './config.guess' 2024-04-28T12:40:40.595 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize: copying file './config.sub' 2024-04-28T12:40:40.611 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize: copying file './install-sh' 2024-04-28T12:40:40.625 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize: copying file './ltmain.sh' 2024-04-28T12:40:40.710 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-28T12:40:40.710 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-28T12:40:40.727 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-28T12:40:40.744 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-28T12:40:40.761 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-28T12:40:40.779 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-28T12:40:40.803 INFO:teuthology.orchestra.run.smithi131.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-28T12:40:40.807 INFO:teuthology.orchestra.run.smithi131.stdout:cp include/install-sh . 2024-04-28T12:40:40.810 INFO:teuthology.orchestra.run.smithi131.stdout:aclocal -I m4 2024-04-28T12:40:42.330 INFO:teuthology.orchestra.run.smithi131.stdout:autoheader 2024-04-28T12:40:42.736 INFO:teuthology.orchestra.run.smithi131.stdout:autoconf 2024-04-28T12:40:42.990 INFO:teuthology.orchestra.run.smithi131.stdout:./configure \ 2024-04-28T12:40:42.990 INFO:teuthology.orchestra.run.smithi131.stdout: --libexecdir=/usr/lib \ 2024-04-28T12:40:42.990 INFO:teuthology.orchestra.run.smithi131.stdout: --exec_prefix=/var/lib 2024-04-28T12:40:43.614 INFO:teuthology.orchestra.run.smithi131.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-28T12:40:43.614 INFO:teuthology.orchestra.run.smithi131.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-28T12:40:43.618 INFO:teuthology.orchestra.run.smithi131.stdout:checking how to print strings... printf 2024-04-28T12:40:43.619 INFO:teuthology.orchestra.run.smithi131.stdout:checking for gcc... gcc 2024-04-28T12:40:43.834 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether the C compiler works... yes 2024-04-28T12:40:43.835 INFO:teuthology.orchestra.run.smithi131.stdout:checking for C compiler default output file name... a.out 2024-04-28T12:40:43.903 INFO:teuthology.orchestra.run.smithi131.stdout:checking for suffix of executables... 2024-04-28T12:40:44.024 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether we are cross compiling... no 2024-04-28T12:40:44.075 INFO:teuthology.orchestra.run.smithi131.stdout:checking for suffix of object files... o 2024-04-28T12:40:44.122 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether we are using the GNU C compiler... yes 2024-04-28T12:40:44.168 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether gcc accepts -g... yes 2024-04-28T12:40:44.268 INFO:teuthology.orchestra.run.smithi131.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-28T12:40:44.276 INFO:teuthology.orchestra.run.smithi131.stdout:checking for a sed that does not truncate output... /bin/sed 2024-04-28T12:40:44.283 INFO:teuthology.orchestra.run.smithi131.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-04-28T12:40:44.287 INFO:teuthology.orchestra.run.smithi131.stdout:checking for egrep... /bin/grep -E 2024-04-28T12:40:44.291 INFO:teuthology.orchestra.run.smithi131.stdout:checking for fgrep... /bin/grep -F 2024-04-28T12:40:44.302 INFO:teuthology.orchestra.run.smithi131.stdout:checking for ld used by gcc... /bin/ld 2024-04-28T12:40:44.308 INFO:teuthology.orchestra.run.smithi131.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-04-28T12:40:44.319 INFO:teuthology.orchestra.run.smithi131.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-04-28T12:40:45.375 INFO:teuthology.orchestra.run.smithi131.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-04-28T12:40:45.375 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether ln -s works... yes 2024-04-28T12:40:45.399 INFO:teuthology.orchestra.run.smithi131.stdout:checking the maximum length of command line arguments... 1572864 2024-04-28T12:40:45.409 INFO:teuthology.orchestra.run.smithi131.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-28T12:40:45.410 INFO:teuthology.orchestra.run.smithi131.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-28T12:40:45.410 INFO:teuthology.orchestra.run.smithi131.stdout:checking for /bin/ld option to reload object files... -r 2024-04-28T12:40:45.411 INFO:teuthology.orchestra.run.smithi131.stdout:checking for objdump... objdump 2024-04-28T12:40:45.412 INFO:teuthology.orchestra.run.smithi131.stdout:checking how to recognize dependent libraries... pass_all 2024-04-28T12:40:45.413 INFO:teuthology.orchestra.run.smithi131.stdout:checking for dlltool... no 2024-04-28T12:40:45.413 INFO:teuthology.orchestra.run.smithi131.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-28T12:40:45.414 INFO:teuthology.orchestra.run.smithi131.stdout:checking for ar... ar 2024-04-28T12:40:45.521 INFO:teuthology.orchestra.run.smithi131.stdout:checking for archiver @FILE support... @ 2024-04-28T12:40:45.522 INFO:teuthology.orchestra.run.smithi131.stdout:checking for strip... strip 2024-04-28T12:40:45.522 INFO:teuthology.orchestra.run.smithi131.stdout:checking for ranlib... ranlib 2024-04-28T12:40:45.523 INFO:teuthology.orchestra.run.smithi131.stdout:checking for gawk... gawk 2024-04-28T12:40:45.696 INFO:teuthology.orchestra.run.smithi131.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-04-28T12:40:45.700 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sysroot... no 2024-04-28T12:40:45.710 INFO:teuthology.orchestra.run.smithi131.stdout:checking for a working dd... /bin/dd 2024-04-28T12:40:45.722 INFO:teuthology.orchestra.run.smithi131.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-04-28T12:40:45.760 INFO:teuthology.orchestra.run.smithi131.stdout:checking for mt... no 2024-04-28T12:40:45.768 INFO:teuthology.orchestra.run.smithi131.stdout:checking if : is a manifest tool... no 2024-04-28T12:40:45.887 INFO:teuthology.orchestra.run.smithi131.stdout:checking how to run the C preprocessor... gcc -E 2024-04-28T12:40:46.161 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:40:46.168 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:40:46.246 INFO:teuthology.orchestra.run.smithi131.stdout:checking for ANSI C header files... yes 2024-04-28T12:40:46.298 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/types.h... yes 2024-04-28T12:40:46.371 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/stat.h... yes 2024-04-28T12:40:46.443 INFO:teuthology.orchestra.run.smithi131.stdout:checking for stdlib.h... yes 2024-04-28T12:40:46.513 INFO:teuthology.orchestra.run.smithi131.stdout:checking for string.h... yes 2024-04-28T12:40:46.585 INFO:teuthology.orchestra.run.smithi131.stdout:checking for memory.h... yes 2024-04-28T12:40:46.655 INFO:teuthology.orchestra.run.smithi131.stdout:checking for strings.h... yes 2024-04-28T12:40:46.728 INFO:teuthology.orchestra.run.smithi131.stdout:checking for inttypes.h... yes 2024-04-28T12:40:46.802 INFO:teuthology.orchestra.run.smithi131.stdout:checking for stdint.h... yes 2024-04-28T12:40:46.883 INFO:teuthology.orchestra.run.smithi131.stdout:checking for unistd.h... yes 2024-04-28T12:40:46.950 INFO:teuthology.orchestra.run.smithi131.stdout:checking for dlfcn.h... yes 2024-04-28T12:40:46.961 INFO:teuthology.orchestra.run.smithi131.stdout:checking for objdir... .libs 2024-04-28T12:40:47.137 INFO:teuthology.orchestra.run.smithi131.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-28T12:40:47.137 INFO:teuthology.orchestra.run.smithi131.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-28T12:40:47.188 INFO:teuthology.orchestra.run.smithi131.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-28T12:40:47.251 INFO:teuthology.orchestra.run.smithi131.stdout:checking if gcc static flag -static works... no 2024-04-28T12:40:47.326 INFO:teuthology.orchestra.run.smithi131.stdout:checking if gcc supports -c -o file.o... yes 2024-04-28T12:40:47.326 INFO:teuthology.orchestra.run.smithi131.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-28T12:40:47.348 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-28T12:40:47.428 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether -lc should be explicitly linked in... no 2024-04-28T12:40:47.573 INFO:teuthology.orchestra.run.smithi131.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-28T12:40:47.573 INFO:teuthology.orchestra.run.smithi131.stdout:checking how to hardcode library paths into programs... immediate 2024-04-28T12:40:47.577 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether stripping libraries is possible... yes 2024-04-28T12:40:47.577 INFO:teuthology.orchestra.run.smithi131.stdout:checking if libtool supports shared libraries... yes 2024-04-28T12:40:47.577 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether to build shared libraries... yes 2024-04-28T12:40:47.577 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether to build static libraries... yes 2024-04-28T12:40:47.598 INFO:teuthology.orchestra.run.smithi131.stdout:checking for gcc... (cached) gcc 2024-04-28T12:40:47.651 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-28T12:40:47.651 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-28T12:40:47.652 INFO:teuthology.orchestra.run.smithi131.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-28T12:40:47.652 INFO:teuthology.orchestra.run.smithi131.stdout:checking for gmake... /usr/bin/gmake 2024-04-28T12:40:47.652 INFO:teuthology.orchestra.run.smithi131.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-28T12:40:47.653 INFO:teuthology.orchestra.run.smithi131.stdout:checking for tar... /bin/tar 2024-04-28T12:40:47.653 INFO:teuthology.orchestra.run.smithi131.stdout:checking for gzip... /bin/gzip 2024-04-28T12:40:47.701 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether gcc -MM is supported... yes 2024-04-28T12:40:47.701 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sort... /bin/sort 2024-04-28T12:40:47.701 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether ln -s works... yes 2024-04-28T12:40:47.702 INFO:teuthology.orchestra.run.smithi131.stdout:checking for rpm... /bin/rpm 2024-04-28T12:40:47.712 INFO:teuthology.orchestra.run.smithi131.stdout:checking for rpmbuild... no 2024-04-28T12:40:47.780 INFO:teuthology.orchestra.run.smithi131.stdout:checking assert.h usability... yes 2024-04-28T12:40:47.808 INFO:teuthology.orchestra.run.smithi131.stdout:checking assert.h presence... yes 2024-04-28T12:40:47.809 INFO:teuthology.orchestra.run.smithi131.stdout:checking for assert.h... yes 2024-04-28T12:40:47.888 INFO:teuthology.orchestra.run.smithi131.stdout:checking bstring.h usability... no 2024-04-28T12:40:47.928 INFO:teuthology.orchestra.run.smithi131.stdout:checking bstring.h presence... no 2024-04-28T12:40:47.928 INFO:teuthology.orchestra.run.smithi131.stdout:checking for bstring.h... no 2024-04-28T12:40:48.000 INFO:teuthology.orchestra.run.smithi131.stdout:checking libgen.h usability... yes 2024-04-28T12:40:48.026 INFO:teuthology.orchestra.run.smithi131.stdout:checking libgen.h presence... yes 2024-04-28T12:40:48.026 INFO:teuthology.orchestra.run.smithi131.stdout:checking for libgen.h... yes 2024-04-28T12:40:48.108 INFO:teuthology.orchestra.run.smithi131.stdout:checking dirent.h usability... yes 2024-04-28T12:40:48.136 INFO:teuthology.orchestra.run.smithi131.stdout:checking dirent.h presence... yes 2024-04-28T12:40:48.136 INFO:teuthology.orchestra.run.smithi131.stdout:checking for dirent.h... yes 2024-04-28T12:40:48.238 INFO:teuthology.orchestra.run.smithi131.stdout:checking errno.h usability... yes 2024-04-28T12:40:48.266 INFO:teuthology.orchestra.run.smithi131.stdout:checking errno.h presence... yes 2024-04-28T12:40:48.266 INFO:teuthology.orchestra.run.smithi131.stdout:checking for errno.h... yes 2024-04-28T12:40:48.346 INFO:teuthology.orchestra.run.smithi131.stdout:checking malloc.h usability... yes 2024-04-28T12:40:48.375 INFO:teuthology.orchestra.run.smithi131.stdout:checking malloc.h presence... yes 2024-04-28T12:40:48.375 INFO:teuthology.orchestra.run.smithi131.stdout:checking for malloc.h... yes 2024-04-28T12:40:48.453 INFO:teuthology.orchestra.run.smithi131.stdout:checking uuid.h usability... no 2024-04-28T12:40:48.493 INFO:teuthology.orchestra.run.smithi131.stdout:checking uuid.h presence... no 2024-04-28T12:40:48.493 INFO:teuthology.orchestra.run.smithi131.stdout:checking for uuid.h... no 2024-04-28T12:40:48.567 INFO:teuthology.orchestra.run.smithi131.stdout:checking uuid/uuid.h usability... yes 2024-04-28T12:40:48.599 INFO:teuthology.orchestra.run.smithi131.stdout:checking uuid/uuid.h presence... yes 2024-04-28T12:40:48.599 INFO:teuthology.orchestra.run.smithi131.stdout:checking for uuid/uuid.h... yes 2024-04-28T12:40:48.680 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/uuid.h usability... no 2024-04-28T12:40:48.719 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/uuid.h presence... no 2024-04-28T12:40:48.720 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/uuid.h... no 2024-04-28T12:40:48.784 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/file.h usability... yes 2024-04-28T12:40:48.812 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/file.h presence... yes 2024-04-28T12:40:48.812 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/file.h... yes 2024-04-28T12:40:48.898 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/fcntl.h usability... yes 2024-04-28T12:40:48.926 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/fcntl.h presence... yes 2024-04-28T12:40:48.926 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/fcntl.h... yes 2024-04-28T12:40:49.007 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/syssgi.h usability... no 2024-04-28T12:40:49.048 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/syssgi.h presence... no 2024-04-28T12:40:49.049 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/syssgi.h... no 2024-04-28T12:40:49.141 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/param.h usability... yes 2024-04-28T12:40:49.174 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/param.h presence... yes 2024-04-28T12:40:49.174 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/param.h... yes 2024-04-28T12:40:49.191 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/stat.h... (cached) yes 2024-04-28T12:40:49.272 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/statvfs.h usability... yes 2024-04-28T12:40:49.301 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/statvfs.h presence... yes 2024-04-28T12:40:49.302 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/statvfs.h... yes 2024-04-28T12:40:49.381 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/time.h usability... yes 2024-04-28T12:40:49.409 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/time.h presence... yes 2024-04-28T12:40:49.409 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/time.h... yes 2024-04-28T12:40:49.493 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/ioctl.h usability... yes 2024-04-28T12:40:49.518 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/ioctl.h presence... yes 2024-04-28T12:40:49.518 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/ioctl.h... yes 2024-04-28T12:40:49.612 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/wait.h usability... yes 2024-04-28T12:40:49.645 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/wait.h presence... yes 2024-04-28T12:40:49.645 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/wait.h... yes 2024-04-28T12:40:49.663 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/types.h... (cached) yes 2024-04-28T12:40:49.681 INFO:teuthology.orchestra.run.smithi131.stdout:checking for strings.h... (cached) yes 2024-04-28T12:40:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:checking err.h usability... yes 2024-04-28T12:40:49.795 INFO:teuthology.orchestra.run.smithi131.stdout:checking err.h presence... yes 2024-04-28T12:40:49.795 INFO:teuthology.orchestra.run.smithi131.stdout:checking for err.h... yes 2024-04-28T12:40:49.874 INFO:teuthology.orchestra.run.smithi131.stdout:checking linux/falloc.h usability... yes 2024-04-28T12:40:49.900 INFO:teuthology.orchestra.run.smithi131.stdout:checking linux/falloc.h presence... yes 2024-04-28T12:40:49.900 INFO:teuthology.orchestra.run.smithi131.stdout:checking for linux/falloc.h... yes 2024-04-28T12:40:49.982 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-28T12:40:50.024 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-28T12:40:50.024 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-28T12:40:50.094 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-28T12:40:50.132 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-28T12:40:50.132 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-28T12:40:50.200 INFO:teuthology.orchestra.run.smithi131.stdout:checking xfs/platform_defs.h usability... no 2024-04-28T12:40:50.237 INFO:teuthology.orchestra.run.smithi131.stdout:checking xfs/platform_defs.h presence... no 2024-04-28T12:40:50.287 INFO:teuthology.orchestra.run.smithi131.stdout:checking for xfs/platform_defs.h... no 2024-04-28T12:40:50.306 INFO:teuthology.orchestra.run.smithi131.stdout:checking btrfs/ioctl.h usability... no 2024-04-28T12:40:50.348 INFO:teuthology.orchestra.run.smithi131.stdout:checking btrfs/ioctl.h presence... no 2024-04-28T12:40:50.348 INFO:teuthology.orchestra.run.smithi131.stdout:checking for btrfs/ioctl.h... no 2024-04-28T12:40:50.419 INFO:teuthology.orchestra.run.smithi131.stdout:checking cifs/ioctl.h usability... no 2024-04-28T12:40:50.461 INFO:teuthology.orchestra.run.smithi131.stdout:checking cifs/ioctl.h presence... no 2024-04-28T12:40:50.462 INFO:teuthology.orchestra.run.smithi131.stdout:checking for cifs/ioctl.h... no 2024-04-28T12:40:50.538 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/mman.h usability... yes 2024-04-28T12:40:50.567 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/mman.h presence... yes 2024-04-28T12:40:50.567 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/mman.h... yes 2024-04-28T12:40:50.646 INFO:teuthology.orchestra.run.smithi131.stdout:checking linux/ext4.h usability... no 2024-04-28T12:40:50.688 INFO:teuthology.orchestra.run.smithi131.stdout:checking linux/ext4.h presence... no 2024-04-28T12:40:50.688 INFO:teuthology.orchestra.run.smithi131.stdout:checking for linux/ext4.h... no 2024-04-28T12:40:50.735 INFO:teuthology.orchestra.run.smithi131.stdout:checking for xfs/xfs_log_format.h... no 2024-04-28T12:40:50.843 INFO:teuthology.orchestra.run.smithi131.stdout:checking for uuid_compare... no 2024-04-28T12:40:50.923 INFO:teuthology.orchestra.run.smithi131.stdout:checking for uuid_compare in -luuid... yes 2024-04-28T12:40:51.041 INFO:teuthology.orchestra.run.smithi131.stdout:checking for xfs/xfs.h... yes 2024-04-28T12:40:51.085 INFO:teuthology.orchestra.run.smithi131.stdout:checking for xfs/libxfs.h... no 2024-04-28T12:40:51.136 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-28T12:40:51.210 INFO:teuthology.orchestra.run.smithi131.stdout:checking for xfs/xqm.h... yes 2024-04-28T12:40:51.320 INFO:teuthology.orchestra.run.smithi131.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-28T12:40:51.380 INFO:teuthology.orchestra.run.smithi131.stdout:checking xfs/handle.h usability... yes 2024-04-28T12:40:51.404 INFO:teuthology.orchestra.run.smithi131.stdout:checking xfs/handle.h presence... yes 2024-04-28T12:40:51.404 INFO:teuthology.orchestra.run.smithi131.stdout:checking for xfs/handle.h... yes 2024-04-28T12:40:51.499 INFO:teuthology.orchestra.run.smithi131.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-28T12:40:51.576 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/acl.h usability... yes 2024-04-28T12:40:51.607 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/acl.h presence... yes 2024-04-28T12:40:51.607 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/acl.h... yes 2024-04-28T12:40:51.693 INFO:teuthology.orchestra.run.smithi131.stdout:checking acl/libacl.h usability... yes 2024-04-28T12:40:51.724 INFO:teuthology.orchestra.run.smithi131.stdout:checking acl/libacl.h presence... yes 2024-04-28T12:40:51.724 INFO:teuthology.orchestra.run.smithi131.stdout:checking for acl/libacl.h... yes 2024-04-28T12:40:51.809 INFO:teuthology.orchestra.run.smithi131.stdout:checking for acl_init in -lacl... yes 2024-04-28T12:40:51.869 INFO:teuthology.orchestra.run.smithi131.stdout:checking gdbm-ndbm.h usability... no 2024-04-28T12:40:51.912 INFO:teuthology.orchestra.run.smithi131.stdout:checking gdbm-ndbm.h presence... no 2024-04-28T12:40:51.912 INFO:teuthology.orchestra.run.smithi131.stdout:checking for gdbm-ndbm.h... no 2024-04-28T12:40:51.968 INFO:teuthology.orchestra.run.smithi131.stdout:checking gdbm/ndbm.h usability... yes 2024-04-28T12:40:51.997 INFO:teuthology.orchestra.run.smithi131.stdout:checking gdbm/ndbm.h presence... yes 2024-04-28T12:40:51.997 INFO:teuthology.orchestra.run.smithi131.stdout:checking for gdbm/ndbm.h... yes 2024-04-28T12:40:52.093 INFO:teuthology.orchestra.run.smithi131.stdout:checking for dbm_open in -lgdbm... no 2024-04-28T12:40:52.183 INFO:teuthology.orchestra.run.smithi131.stdout:checking for dbm_fetch in -lgdbm... no 2024-04-28T12:40:52.273 INFO:teuthology.orchestra.run.smithi131.stdout:checking for dbm_store in -lgdbm... no 2024-04-28T12:40:52.368 INFO:teuthology.orchestra.run.smithi131.stdout:checking for dbm_close in -lgdbm... no 2024-04-28T12:40:52.461 INFO:teuthology.orchestra.run.smithi131.stdout:checking for dbm_open in -lgdbm_compat... yes 2024-04-28T12:40:52.542 INFO:teuthology.orchestra.run.smithi131.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2024-04-28T12:40:52.621 INFO:teuthology.orchestra.run.smithi131.stdout:checking for dbm_store in -lgdbm_compat... yes 2024-04-28T12:40:52.703 INFO:teuthology.orchestra.run.smithi131.stdout:checking for dbm_close in -lgdbm_compat... yes 2024-04-28T12:40:52.764 INFO:teuthology.orchestra.run.smithi131.stdout:checking libaio.h usability... yes 2024-04-28T12:40:52.794 INFO:teuthology.orchestra.run.smithi131.stdout:checking libaio.h presence... yes 2024-04-28T12:40:52.794 INFO:teuthology.orchestra.run.smithi131.stdout:checking for libaio.h... yes 2024-04-28T12:40:52.863 INFO:teuthology.orchestra.run.smithi131.stdout:checking liburing.h usability... no 2024-04-28T12:40:52.906 INFO:teuthology.orchestra.run.smithi131.stdout:checking liburing.h presence... no 2024-04-28T12:40:52.906 INFO:teuthology.orchestra.run.smithi131.stdout:checking for liburing.h... no 2024-04-28T12:40:52.980 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/capability.h usability... yes 2024-04-28T12:40:53.011 INFO:teuthology.orchestra.run.smithi131.stdout:checking sys/capability.h presence... yes 2024-04-28T12:40:53.011 INFO:teuthology.orchestra.run.smithi131.stdout:checking for sys/capability.h... yes 2024-04-28T12:40:53.070 INFO:teuthology.orchestra.run.smithi131.stdout:checking linux/fiemap.h usability... yes 2024-04-28T12:40:53.093 INFO:teuthology.orchestra.run.smithi131.stdout:checking linux/fiemap.h presence... yes 2024-04-28T12:40:53.093 INFO:teuthology.orchestra.run.smithi131.stdout:checking for linux/fiemap.h... yes 2024-04-28T12:40:53.173 INFO:teuthology.orchestra.run.smithi131.stdout:checking for fallocate... yes 2024-04-28T12:40:53.259 INFO:teuthology.orchestra.run.smithi131.stdout:checking for open_by_handle_at... yes 2024-04-28T12:40:53.319 INFO:teuthology.orchestra.run.smithi131.stdout:checking linux/fs.h usability... yes 2024-04-28T12:40:53.344 INFO:teuthology.orchestra.run.smithi131.stdout:checking linux/fs.h presence... yes 2024-04-28T12:40:53.345 INFO:teuthology.orchestra.run.smithi131.stdout:checking for linux/fs.h... yes 2024-04-28T12:40:53.408 INFO:teuthology.orchestra.run.smithi131.stdout:checking btrfsutil.h usability... no 2024-04-28T12:40:53.451 INFO:teuthology.orchestra.run.smithi131.stdout:checking btrfsutil.h presence... no 2024-04-28T12:40:53.451 INFO:teuthology.orchestra.run.smithi131.stdout:checking for btrfsutil.h... no 2024-04-28T12:40:53.554 INFO:teuthology.orchestra.run.smithi131.stdout:checking for copy_file_range... yes 2024-04-28T12:40:53.639 INFO:teuthology.orchestra.run.smithi131.stdout:checking for SEEK_DATA... yes 2024-04-28T12:40:53.753 INFO:teuthology.orchestra.run.smithi131.stdout:checking for BMV_OF_SHARED... yes 2024-04-28T12:40:53.842 INFO:teuthology.orchestra.run.smithi131.stdout:checking for nftw... yes 2024-04-28T12:40:53.946 INFO:teuthology.orchestra.run.smithi131.stdout:checking for RLIMIT_NOFILE... yes 2024-04-28T12:40:54.024 INFO:teuthology.orchestra.run.smithi131.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-28T12:40:54.112 INFO:teuthology.orchestra.run.smithi131.stdout:checking for FICLONE... yes 2024-04-28T12:40:54.195 INFO:teuthology.orchestra.run.smithi131.stdout:checking for renameat2... yes 2024-04-28T12:40:54.280 INFO:teuthology.orchestra.run.smithi131.stdout:checking for reallocarray... yes 2024-04-28T12:40:54.377 INFO:teuthology.orchestra.run.smithi131.stdout:checking for struct mount_attr... yes 2024-04-28T12:40:54.483 INFO:teuthology.orchestra.run.smithi131.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-28T12:40:54.584 INFO:teuthology.orchestra.run.smithi131.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-28T12:40:54.687 INFO:teuthology.orchestra.run.smithi131.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-28T12:40:54.794 INFO:teuthology.orchestra.run.smithi131.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-28T12:40:54.904 INFO:teuthology.orchestra.run.smithi131.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-28T12:40:55.016 INFO:teuthology.orchestra.run.smithi131.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-28T12:40:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-28T12:40:55.190 INFO:teuthology.orchestra.run.smithi131.stdout:checking linux/btrfs.h usability... yes 2024-04-28T12:40:55.219 INFO:teuthology.orchestra.run.smithi131.stdout:checking linux/btrfs.h presence... yes 2024-04-28T12:40:55.219 INFO:teuthology.orchestra.run.smithi131.stdout:checking for linux/btrfs.h... yes 2024-04-28T12:40:55.300 INFO:teuthology.orchestra.run.smithi131.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-28T12:40:55.328 INFO:teuthology.orchestra.run.smithi131.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-28T12:40:55.329 INFO:teuthology.orchestra.run.smithi131.stdout:checking for linux/btrfs_tree.h... yes 2024-04-28T12:40:55.392 INFO:teuthology.orchestra.run.smithi131.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-28T12:40:55.460 INFO:teuthology.orchestra.run.smithi131.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-28T12:40:55.494 INFO:teuthology.orchestra.run.smithi131.stdout:configure: creating ./config.status 2024-04-28T12:40:56.703 INFO:teuthology.orchestra.run.smithi131.stdout:config.status: creating include/builddefs 2024-04-28T12:40:56.744 INFO:teuthology.orchestra.run.smithi131.stdout:config.status: creating include/config.h 2024-04-28T12:40:56.775 INFO:teuthology.orchestra.run.smithi131.stdout:config.status: executing libtool commands 2024-04-28T12:40:56.877 INFO:teuthology.orchestra.run.smithi131.stdout:Building include 2024-04-28T12:40:56.888 INFO:teuthology.orchestra.run.smithi131.stdout:Building lib 2024-04-28T12:40:56.897 INFO:teuthology.orchestra.run.smithi131.stdout: [LTDEP] 2024-04-28T12:40:57.222 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] dataascii.lo 2024-04-28T12:40:57.506 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] databin.lo 2024-04-28T12:40:57.796 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] datapid.lo 2024-04-28T12:40:58.039 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] file_lock.lo 2024-04-28T12:40:58.337 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] forker.lo 2024-04-28T12:40:58.601 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] pattern.lo 2024-04-28T12:40:58.861 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] open_flags.lo 2024-04-28T12:40:59.152 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] random_range.lo 2024-04-28T12:40:59.526 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] string_to_tokens.lo 2024-04-28T12:40:59.754 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] str_to_bytes.lo 2024-04-28T12:41:00.045 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] tlibio.lo 2024-04-28T12:41:00.441 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] write_log.lo 2024-04-28T12:41:00.820 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] random.lo 2024-04-28T12:41:01.064 INFO:teuthology.orchestra.run.smithi131.stdout: [LD] 2024-04-28T12:41:01.273 INFO:teuthology.orchestra.run.smithi131.stdout:Building ltp 2024-04-28T12:41:01.283 INFO:teuthology.orchestra.run.smithi131.stdout: [DEP] 2024-04-28T12:41:01.584 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] doio 2024-04-28T12:41:02.486 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fsstress 2024-04-28T12:41:04.104 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fsx 2024-04-28T12:41:05.002 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] growfiles 2024-04-28T12:41:05.729 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] iogen 2024-04-28T12:41:06.266 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-stress 2024-04-28T12:41:06.847 INFO:teuthology.orchestra.run.smithi131.stdout:Building src 2024-04-28T12:41:06.859 INFO:teuthology.orchestra.run.smithi131.stdout: [DEP] 2024-04-28T12:41:10.769 INFO:teuthology.orchestra.run.smithi131.stdout: [DEP] 2024-04-28T12:41:10.844 INFO:teuthology.orchestra.run.smithi131.stdout: [DEP] 2024-04-28T12:41:10.866 INFO:teuthology.orchestra.run.smithi131.stdout: [DEP] 2024-04-28T12:41:11.347 INFO:teuthology.orchestra.run.smithi131.stdout: [DEP] 2024-04-28T12:41:11.402 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] dirstress 2024-04-28T12:41:11.835 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fill 2024-04-28T12:41:12.162 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fill2 2024-04-28T12:41:12.567 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] getpagesize 2024-04-28T12:41:12.848 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] holes 2024-04-28T12:41:13.226 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] lstat64 2024-04-28T12:41:13.573 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] nametest 2024-04-28T12:41:13.994 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] permname 2024-04-28T12:41:14.352 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] randholes 2024-04-28T12:41:14.779 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] runas 2024-04-28T12:41:15.145 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] truncfile 2024-04-28T12:41:15.498 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] usemem 2024-04-28T12:41:15.821 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] mmapcat 2024-04-28T12:41:16.144 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] append_reader 2024-04-28T12:41:16.289 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:41:16.295 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:41:16.455 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] append_writer 2024-04-28T12:41:16.722 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] dirperf 2024-04-28T12:41:17.115 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] metaperf 2024-04-28T12:41:17.540 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] devzero 2024-04-28T12:41:17.831 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] feature 2024-04-28T12:41:18.163 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] alloc 2024-04-28T12:41:18.534 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fault 2024-04-28T12:41:18.873 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fstest 2024-04-28T12:41:19.267 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_access_root 2024-04-28T12:41:19.594 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] godown 2024-04-28T12:41:19.907 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] resvtest 2024-04-28T12:41:20.243 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] writemod 2024-04-28T12:41:20.526 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] writev_on_pagefault 2024-04-28T12:41:20.856 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] makeextents 2024-04-28T12:41:21.207 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] itrash 2024-04-28T12:41:21.527 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] rename 2024-04-28T12:41:21.819 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] multi_open_unlink 2024-04-28T12:41:22.130 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] unwritten_sync 2024-04-28T12:41:22.490 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] genhashnames 2024-04-28T12:41:22.828 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_holes 2024-04-28T12:41:23.142 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_mmap_writev 2024-04-28T12:41:23.462 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_truncate_cmtime 2024-04-28T12:41:23.764 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] dirhash_collide 2024-04-28T12:41:24.113 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_rename_overwrite 2024-04-28T12:41:24.406 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] holetest 2024-04-28T12:41:24.814 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_truncate_self 2024-04-28T12:41:25.101 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] af_unix 2024-04-28T12:41:25.396 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_mmap_stale_pmd 2024-04-28T12:41:25.712 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_mmap_cow_race 2024-04-28T12:41:26.054 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_mmap_fallocate 2024-04-28T12:41:26.371 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fsync-err 2024-04-28T12:41:26.716 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_mmap_write_ro 2024-04-28T12:41:27.041 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-28T12:41:27.346 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-28T12:41:27.632 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_ofd_locks 2024-04-28T12:41:27.987 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_mmap_collision 2024-04-28T12:41:28.362 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] mmap-write-concurrent 2024-04-28T12:41:28.699 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_get_file_time 2024-04-28T12:41:29.001 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_create_short_dirs 2024-04-28T12:41:29.354 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_create_long_dirs 2024-04-28T12:41:29.676 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_enospc 2024-04-28T12:41:30.046 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_mmap_writev_overlap 2024-04-28T12:41:30.391 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] checkpoint_journal 2024-04-28T12:41:30.705 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] mmap-rw-fault 2024-04-28T12:41:31.039 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] allocstale 2024-04-28T12:41:31.373 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_mmap_cow_memory_failure 2024-04-28T12:41:31.724 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fake-dump-rootino 2024-04-28T12:41:32.055 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] dio-buf-fault 2024-04-28T12:41:32.357 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] rewinddir-test 2024-04-28T12:41:32.696 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] readdir-while-renames 2024-04-28T12:41:33.004 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] xfsctl 2024-04-28T12:41:33.360 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] bstat 2024-04-28T12:41:33.750 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_mtab 2024-04-28T12:41:34.113 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] getdevicesize 2024-04-28T12:41:34.398 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] preallo_rw_pattern_reader 2024-04-28T12:41:34.737 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] preallo_rw_pattern_writer 2024-04-28T12:41:35.046 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] ftrunc 2024-04-28T12:41:35.380 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] trunc 2024-04-28T12:41:35.716 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fs_perms 2024-04-28T12:41:36.054 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] testx 2024-04-28T12:41:36.343 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] looptest 2024-04-28T12:41:36.679 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] locktest 2024-04-28T12:41:37.169 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] unwritten_mmap 2024-04-28T12:41:37.509 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] bulkstat_unlink_test 2024-04-28T12:41:37.870 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] deduperace 2024-04-28T12:41:38.242 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] bulkstat_unlink_test_modified 2024-04-28T12:41:38.617 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_dir_offset 2024-04-28T12:41:38.939 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_futimens 2024-04-28T12:41:39.250 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_immutable 2024-04-28T12:41:40.082 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] stale_handle 2024-04-28T12:41:40.399 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] pwrite_mmap_blocked 2024-04-28T12:41:40.714 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_dir_offset2 2024-04-28T12:41:41.056 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] seek_sanity_test 2024-04-28T12:41:41.658 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] seek_copy_test 2024-04-28T12:41:42.002 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_readdir_1 2024-04-28T12:41:42.305 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_readdir_2 2024-04-28T12:41:42.621 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fsync-tester 2024-04-28T12:41:43.063 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] nsexec 2024-04-28T12:41:43.408 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] cloner 2024-04-28T12:41:43.759 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] renameat2 2024-04-28T12:41:44.109 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_getcwd 2024-04-28T12:41:44.438 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] e4compact 2024-04-28T12:41:44.821 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] test-nextquota 2024-04-28T12:41:45.184 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] punch-alternating 2024-04-28T12:41:45.538 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-28T12:41:45.863 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] listxattr 2024-04-28T12:41:46.181 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] dio-interleaved 2024-04-28T12:41:46.408 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:41:46.415 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:41:46.520 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_dir_type 2024-04-28T12:41:46.844 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] dio-invalidate-cache 2024-04-28T12:41:47.217 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] stat_test 2024-04-28T12:41:47.711 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_encrypted_d_revalidate 2024-04-28T12:41:48.043 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] attr_replace_test 2024-04-28T12:41:48.358 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] swapon 2024-04-28T12:41:48.674 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] mkswap 2024-04-28T12:41:48.994 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_attr_corruption 2024-04-28T12:41:49.323 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_open_tmpfiles 2024-04-28T12:41:49.689 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fscrypt-crypt-util 2024-04-28T12:41:50.564 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] bulkstat_null_ocount 2024-04-28T12:41:50.856 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] splice-test 2024-04-28T12:41:51.210 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] chprojid_fail 2024-04-28T12:41:51.546 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] detached_mounts_propagation 2024-04-28T12:41:51.919 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] ext4_resize 2024-04-28T12:41:52.217 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_readdir_3 2024-04-28T12:41:52.581 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] splice2pipe 2024-04-28T12:41:52.917 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] uuid_ioctl 2024-04-28T12:41:53.255 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-28T12:41:53.589 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fiemap-fault 2024-04-28T12:41:53.894 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fiemap-tester 2024-04-28T12:41:54.342 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_stripealign 2024-04-28T12:41:54.665 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] open_by_handle 2024-04-28T12:41:55.097 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] dbtest 2024-04-28T12:41:55.506 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_mmap_dio 2024-04-28T12:41:55.842 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] xfsfind 2024-04-28T12:41:56.204 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] t_reflink_read_race 2024-04-28T12:41:56.556 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] fssum 2024-04-28T12:41:57.203 INFO:teuthology.orchestra.run.smithi131.stdout:Building log-writes 2024-04-28T12:41:57.216 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] replay-log 2024-04-28T12:41:57.724 INFO:teuthology.orchestra.run.smithi131.stdout:Building perf 2024-04-28T12:41:57.734 INFO:teuthology.orchestra.run.smithi131.stdout:Building aio-dio-regress 2024-04-28T12:41:57.748 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aiocp 2024-04-28T12:41:58.126 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-28T12:41:58.440 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-dio-append-write-read-race 2024-04-28T12:41:58.791 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-dio-cow-race 2024-04-28T12:41:59.117 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-dio-cycle-write 2024-04-28T12:41:59.439 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-dio-eof-race 2024-04-28T12:41:59.765 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-dio-extend-stat 2024-04-28T12:42:00.092 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-dio-fcntl-race 2024-04-28T12:42:00.417 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-dio-hole-filling-race 2024-04-28T12:42:00.734 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-dio-invalidate-failure 2024-04-28T12:42:01.069 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-dio-invalidate-readahead 2024-04-28T12:42:01.406 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aiodio_sparse2 2024-04-28T12:42:01.805 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-dio-subblock-eof-read 2024-04-28T12:42:02.104 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-dio-write-verify 2024-04-28T12:42:02.447 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-28T12:42:02.723 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-28T12:42:03.006 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] aio-last-ref-held-by-io 2024-04-28T12:42:03.364 INFO:teuthology.orchestra.run.smithi131.stdout:Building vfs 2024-04-28T12:42:03.378 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] vfstest 2024-04-28T12:42:07.332 INFO:teuthology.orchestra.run.smithi131.stdout: [CC] mount-idmapped 2024-04-28T12:42:07.892 INFO:teuthology.orchestra.run.smithi131.stdout:Building m4 2024-04-28T12:42:07.902 INFO:teuthology.orchestra.run.smithi131.stdout:Building common 2024-04-28T12:42:07.912 INFO:teuthology.orchestra.run.smithi131.stdout:Building tests 2024-04-28T12:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout:Building btrfs 2024-04-28T12:42:07.943 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/btrfs/group.list 2024-04-28T12:42:08.157 INFO:teuthology.orchestra.run.smithi131.stdout:Building ceph 2024-04-28T12:42:08.171 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/ceph/group.list 2024-04-28T12:42:08.221 INFO:teuthology.orchestra.run.smithi131.stdout:Building cifs 2024-04-28T12:42:08.233 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/cifs/group.list 2024-04-28T12:42:08.276 INFO:teuthology.orchestra.run.smithi131.stdout:Building ext4 2024-04-28T12:42:08.291 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/ext4/group.list 2024-04-28T12:42:08.431 INFO:teuthology.orchestra.run.smithi131.stdout:Building f2fs 2024-04-28T12:42:08.447 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/f2fs/group.list 2024-04-28T12:42:08.511 INFO:teuthology.orchestra.run.smithi131.stdout:Building generic 2024-04-28T12:42:08.528 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/generic/group.list 2024-04-28T12:42:08.895 INFO:teuthology.orchestra.run.smithi131.stdout:Building nfs 2024-04-28T12:42:08.910 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/nfs/group.list 2024-04-28T12:42:08.962 INFO:teuthology.orchestra.run.smithi131.stdout:Building ocfs2 2024-04-28T12:42:08.975 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/ocfs2/group.list 2024-04-28T12:42:09.018 INFO:teuthology.orchestra.run.smithi131.stdout:Building overlay 2024-04-28T12:42:09.033 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/overlay/group.list 2024-04-28T12:42:09.183 INFO:teuthology.orchestra.run.smithi131.stdout:Building perf 2024-04-28T12:42:09.197 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/perf/group.list 2024-04-28T12:42:09.238 INFO:teuthology.orchestra.run.smithi131.stdout:Building selftest 2024-04-28T12:42:09.251 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/selftest/group.list 2024-04-28T12:42:09.294 INFO:teuthology.orchestra.run.smithi131.stdout:Building shared 2024-04-28T12:42:09.309 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/shared/group.list 2024-04-28T12:42:09.374 INFO:teuthology.orchestra.run.smithi131.stdout:Building tmpfs 2024-04-28T12:42:09.388 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/tmpfs/group.list 2024-04-28T12:42:09.437 INFO:teuthology.orchestra.run.smithi131.stdout:Building udf 2024-04-28T12:42:09.452 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/udf/group.list 2024-04-28T12:42:09.492 INFO:teuthology.orchestra.run.smithi131.stdout:Building xfs 2024-04-28T12:42:09.510 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/xfs/group.list 2024-04-28T12:42:09.871 DEBUG:teuthology.orchestra.run.smithi131:> (cd /tmp/tmp.IDf8sLWWdqxfstests-dev && exec sudo make install) 2024-04-28T12:42:09.986 INFO:teuthology.orchestra.run.smithi131.stdout:Building include 2024-04-28T12:42:09.997 INFO:teuthology.orchestra.run.smithi131.stdout:Building lib 2024-04-28T12:42:10.008 INFO:teuthology.orchestra.run.smithi131.stdout:Building ltp 2024-04-28T12:42:10.019 INFO:teuthology.orchestra.run.smithi131.stdout:Building src 2024-04-28T12:42:10.106 INFO:teuthology.orchestra.run.smithi131.stdout:Building log-writes 2024-04-28T12:42:10.116 INFO:teuthology.orchestra.run.smithi131.stdout:Building perf 2024-04-28T12:42:10.127 INFO:teuthology.orchestra.run.smithi131.stdout:Building aio-dio-regress 2024-04-28T12:42:10.139 INFO:teuthology.orchestra.run.smithi131.stdout:Building vfs 2024-04-28T12:42:10.150 INFO:teuthology.orchestra.run.smithi131.stdout:Building m4 2024-04-28T12:42:10.160 INFO:teuthology.orchestra.run.smithi131.stdout:Building common 2024-04-28T12:42:10.171 INFO:teuthology.orchestra.run.smithi131.stdout:Building tests 2024-04-28T12:42:10.186 INFO:teuthology.orchestra.run.smithi131.stdout:Building btrfs 2024-04-28T12:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/btrfs/group.list 2024-04-28T12:42:10.418 INFO:teuthology.orchestra.run.smithi131.stdout:Building ceph 2024-04-28T12:42:10.432 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/ceph/group.list 2024-04-28T12:42:10.486 INFO:teuthology.orchestra.run.smithi131.stdout:Building cifs 2024-04-28T12:42:10.501 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/cifs/group.list 2024-04-28T12:42:10.546 INFO:teuthology.orchestra.run.smithi131.stdout:Building ext4 2024-04-28T12:42:10.561 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/ext4/group.list 2024-04-28T12:42:10.700 INFO:teuthology.orchestra.run.smithi131.stdout:Building f2fs 2024-04-28T12:42:10.714 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/f2fs/group.list 2024-04-28T12:42:10.772 INFO:teuthology.orchestra.run.smithi131.stdout:Building generic 2024-04-28T12:42:10.787 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/generic/group.list 2024-04-28T12:42:11.122 INFO:teuthology.orchestra.run.smithi131.stdout:Building nfs 2024-04-28T12:42:11.136 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/nfs/group.list 2024-04-28T12:42:11.188 INFO:teuthology.orchestra.run.smithi131.stdout:Building ocfs2 2024-04-28T12:42:11.202 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/ocfs2/group.list 2024-04-28T12:42:11.252 INFO:teuthology.orchestra.run.smithi131.stdout:Building overlay 2024-04-28T12:42:11.267 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/overlay/group.list 2024-04-28T12:42:11.410 INFO:teuthology.orchestra.run.smithi131.stdout:Building perf 2024-04-28T12:42:11.424 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/perf/group.list 2024-04-28T12:42:11.465 INFO:teuthology.orchestra.run.smithi131.stdout:Building selftest 2024-04-28T12:42:11.479 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/selftest/group.list 2024-04-28T12:42:11.522 INFO:teuthology.orchestra.run.smithi131.stdout:Building shared 2024-04-28T12:42:11.537 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/shared/group.list 2024-04-28T12:42:11.599 INFO:teuthology.orchestra.run.smithi131.stdout:Building tmpfs 2024-04-28T12:42:11.614 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/tmpfs/group.list 2024-04-28T12:42:11.664 INFO:teuthology.orchestra.run.smithi131.stdout:Building udf 2024-04-28T12:42:11.678 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/udf/group.list 2024-04-28T12:42:11.718 INFO:teuthology.orchestra.run.smithi131.stdout:Building xfs 2024-04-28T12:42:11.735 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/xfs/group.list 2024-04-28T12:42:12.073 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-04-28T12:42:12.076 INFO:teuthology.orchestra.run.smithi131.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-28T12:42:12.077 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-04-28T12:42:12.082 INFO:teuthology.orchestra.run.smithi131.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-28T12:42:12.082 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-04-28T12:42:12.087 INFO:teuthology.orchestra.run.smithi131.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-04-28T12:42:12.120 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-04-28T12:42:12.355 INFO:teuthology.orchestra.run.smithi131.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-04-28T12:42:12.388 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-04-28T12:42:12.433 INFO:teuthology.orchestra.run.smithi131.stdout:Building log-writes 2024-04-28T12:42:12.443 INFO:teuthology.orchestra.run.smithi131.stdout:Building perf 2024-04-28T12:42:12.454 INFO:teuthology.orchestra.run.smithi131.stdout:Building aio-dio-regress 2024-04-28T12:42:12.466 INFO:teuthology.orchestra.run.smithi131.stdout:Building vfs 2024-04-28T12:42:12.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake -C log-writes install 2024-04-28T12:42:12.478 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-04-28T12:42:12.509 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-04-28T12:42:12.542 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake -C perf install 2024-04-28T12:42:12.546 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-04-28T12:42:12.579 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-04-28T12:42:12.658 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-04-28T12:42:12.663 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-04-28T12:42:12.694 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-04-28T12:42:12.879 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake -C vfs install 2024-04-28T12:42:12.883 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-04-28T12:42:12.913 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-04-28T12:42:12.957 INFO:teuthology.orchestra.run.smithi131.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-04-28T12:42:12.984 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-04-28T12:42:16.558 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:42:16.564 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:42:17.337 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-04-28T12:42:17.682 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-04-28T12:42:17.749 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-04-28T12:42:17.753 INFO:teuthology.orchestra.run.smithi131.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-28T12:42:17.753 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-04-28T12:42:17.757 INFO:teuthology.orchestra.run.smithi131.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-04-28T12:42:17.789 INFO:teuthology.orchestra.run.smithi131.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-04-28T12:42:18.356 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-04-28T12:42:18.360 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/btrfs/ install 2024-04-28T12:42:18.368 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/btrfs/group.list 2024-04-28T12:42:18.579 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-04-28T12:42:18.610 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 320 330 /var/lib/xfstests/tests/btrfs 2024-04-28T12:42:21.729 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-04-28T12:42:21.762 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 320.out 330.out /var/lib/xfstests/tests/btrfs 2024-04-28T12:42:25.065 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/ceph/ install 2024-04-28T12:42:25.072 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/ceph/group.list 2024-04-28T12:42:25.118 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-04-28T12:42:25.147 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-04-28T12:42:25.214 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-04-28T12:42:25.241 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-04-28T12:42:25.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/cifs/ install 2024-04-28T12:42:25.315 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/cifs/group.list 2024-04-28T12:42:25.356 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-04-28T12:42:25.386 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-04-28T12:42:25.417 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-04-28T12:42:25.448 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-04-28T12:42:25.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/ext4/ install 2024-04-28T12:42:25.485 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/ext4/group.list 2024-04-28T12:42:25.621 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-04-28T12:42:25.652 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-04-28T12:42:26.313 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-04-28T12:42:26.339 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-04-28T12:42:27.078 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/f2fs/ install 2024-04-28T12:42:27.085 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/f2fs/group.list 2024-04-28T12:42:27.146 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-04-28T12:42:27.178 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-04-28T12:42:27.220 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-04-28T12:42:27.252 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-04-28T12:42:27.294 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/generic/ install 2024-04-28T12:42:27.303 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/generic/group.list 2024-04-28T12:42:27.674 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-04-28T12:42:27.708 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 741 742 743 /var/lib/xfstests/tests/generic 2024-04-28T12:42:35.731 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-04-28T12:42:35.763 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 741.out 742.out 743.out /var/lib/xfstests/tests/generic 2024-04-28T12:42:43.678 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/nfs/ install 2024-04-28T12:42:43.685 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/nfs/group.list 2024-04-28T12:42:43.735 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-04-28T12:42:43.767 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-04-28T12:42:43.799 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-04-28T12:42:43.830 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-04-28T12:42:43.863 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/ocfs2/ install 2024-04-28T12:42:43.870 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/ocfs2/group.list 2024-04-28T12:42:43.916 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-04-28T12:42:43.948 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-04-28T12:42:43.980 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-04-28T12:42:44.011 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-04-28T12:42:44.043 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/overlay/ install 2024-04-28T12:42:44.050 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/overlay/group.list 2024-04-28T12:42:44.196 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-04-28T12:42:44.228 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-04-28T12:42:45.335 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-04-28T12:42:45.368 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-04-28T12:42:46.475 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/perf/ install 2024-04-28T12:42:46.482 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/perf/group.list 2024-04-28T12:42:46.520 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-04-28T12:42:46.550 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-04-28T12:42:46.581 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-04-28T12:42:46.613 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-04-28T12:42:46.645 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/selftest/ install 2024-04-28T12:42:46.652 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/selftest/group.list 2024-04-28T12:42:46.694 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-04-28T12:42:46.712 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:42:46.718 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:42:46.726 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-04-28T12:42:46.794 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-04-28T12:42:46.822 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-04-28T12:42:46.901 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/shared/ install 2024-04-28T12:42:46.908 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/shared/group.list 2024-04-28T12:42:46.966 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-04-28T12:42:46.997 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-04-28T12:42:47.047 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-04-28T12:42:47.080 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-04-28T12:42:47.134 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/tmpfs/ install 2024-04-28T12:42:47.141 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/tmpfs/group.list 2024-04-28T12:42:47.183 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-04-28T12:42:47.213 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-04-28T12:42:47.243 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-04-28T12:42:47.274 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-04-28T12:42:47.305 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/udf/ install 2024-04-28T12:42:47.312 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/udf/group.list 2024-04-28T12:42:47.350 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-04-28T12:42:47.381 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-04-28T12:42:47.412 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-04-28T12:42:47.442 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-04-28T12:42:47.474 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/xfs/ install 2024-04-28T12:42:47.482 INFO:teuthology.orchestra.run.smithi131.stdout: [GROUP] /tmp/tmp.IDf8sLWWdqxfstests-dev/tests/xfs/group.list 2024-04-28T12:42:47.819 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-04-28T12:42:47.850 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-04-28T12:42:55.355 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-04-28T12:42:55.390 INFO:teuthology.orchestra.run.smithi131.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-04-28T12:43:02.853 INFO:teuthology.orchestra.run.smithi131.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-04-28T12:43:02.883 INFO:teuthology.orchestra.run.smithi131.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-04-28T12:43:02.913 INFO:teuthology.orchestra.run.smithi131.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-04-28T12:43:02.946 INFO:teuthology.orchestra.run.smithi131.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-04-28T12:43:02.982 INFO:tasks.cephfs.test_acls:client is fuse mounted 2024-04-28T12:43:02.982 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-28T12:43:02.982 DEBUG:teuthology.orchestra.run.smithi131:running tests for ACLs from xfstests-dev> (cd /tmp/tmp.IDf8sLWWdqxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-04-28T12:43:03.536 INFO:teuthology.orchestra.run.smithi131.stdout:FSTYP -- ceph-fuse 2024-04-28T12:43:03.552 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM -- Linux/x86_64 smithi131 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-28T12:43:03.552 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:04.600 INFO:teuthology.orchestra.run.smithi131.stdout:generic/099 1s 2024-04-28T12:43:04.617 INFO:teuthology.orchestra.run.smithi131.stdout:Ran: generic/099 2024-04-28T12:43:04.617 INFO:teuthology.orchestra.run.smithi131.stdout:Passed all 1 tests 2024-04-28T12:43:04.625 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:04.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[49497]: fuse finished with error 0 and tester_r 0 2024-04-28T12:43:04.654 INFO:tasks.cephfs.test_acls:Command return value: 0 2024-04-28T12:43:04.654 INFO:tasks.cephfs.test_acls:Command stdout - FSTYP -- ceph-fuse PLATFORM -- Linux/x86_64 smithi131 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 generic/099 1s Ran: generic/099 Passed all 1 tests 2024-04-28T12:43:04.654 INFO:tasks.cephfs.test_acls:Command stderr - 2024-04-28T12:43:04.655 DEBUG:teuthology.orchestra.run.smithi131:> sudo userdel --force --remove fsgqa 2024-04-28T12:43:05.339 DEBUG:teuthology.orchestra.run.smithi131:> sudo userdel --force --remove 123456-fsgqa 2024-04-28T12:43:06.008 DEBUG:teuthology.orchestra.run.smithi131:> sudo groupdel fsgqa 2024-04-28T12:43:06.102 INFO:teuthology.orchestra.run.smithi131.stderr:groupdel: group 'fsgqa' does not exist 2024-04-28T12:43:06.107 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-28T12:43:06.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:43:06.108 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo stat /tmp/tmp.IDf8sLWWdqxfstests-dev/results) 2024-04-28T12:43:06.158 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: cd: /home/ubuntu/cephtest/mnt.0: Permission denied 2024-04-28T12:43:06.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:43:06.160 INFO:tasks.cephfs.xfstests_dev:xfstests-dev repo contains not directory named "results". repo location: /tmp/tmp.IDf8sLWWdqxfstests-dev 2024-04-28T12:43:06.160 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf /tmp/tmp.IDf8sLWWdqxfstests-dev 2024-04-28T12:43:06.478 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-28T12:43:06.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-28T12:43:06.581 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-28T12:43:06.690 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-28T12:43:06.690 INFO:tasks.cephfs.mount:Terminating background process 2024-04-28T12:43:06.690 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:43:06.690 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:43:06.748 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-28T12:43:06.748 INFO:tasks.cephfs.mount:Terminating background process 2024-04-28T12:43:06.749 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:43:06.749 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:43:06.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-28T12:43:06.806 INFO:tasks.cephfs.mount:Terminating background process 2024-04-28T12:43:06.806 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:43:06.806 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:43:06.863 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-28T12:43:06.863 INFO:tasks.cephfs.mount:Terminating background process 2024-04-28T12:43:06.863 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:43:06.864 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:43:06.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-28T12:43:06.922 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:43:07.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.132+0000 7fa0dd400640 1 Processor -- start 2024-04-28T12:43:07.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.132+0000 7fa0dd400640 1 -- start start 2024-04-28T12:43:07.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.133+0000 7fa0dd400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fa0d81467c0 0x7fa0d8072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.133+0000 7fa0dd400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa0d80b3ce0 0x7fa0d80d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.133+0000 7fa0dd400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d80b4140 0x7fa0d80d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.133+0000 7fa0dd400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa0d8073dd0 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.133+0000 7fa0dd400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fa0d8146ba0 con 0x7fa0d81467c0 2024-04-28T12:43:07.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.133+0000 7fa0dd400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d80731c0 con 0x7fa0d80b4140 2024-04-28T12:43:07.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.133+0000 7fa0cde00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa0d80b3ce0 0x7fa0d80d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:07.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.133+0000 7fa0cde00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa0d80b3ce0 0x7fa0d80d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53826/0 (socket says 172.21.15.131:53826) 2024-04-28T12:43:07.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.134+0000 7fa0cde00640 1 -- 172.21.15.131:0/3639696173 learned_addr learned my addr 172.21.15.131:0/3639696173 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:07.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.134+0000 7fa0d6800640 1 --2- 172.21.15.131:0/3639696173 >> v2:172.21.15.157:3300/0 conn(0x7fa0d81467c0 0x7fa0d8072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:07.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.134+0000 7fa0cde00640 1 -- 172.21.15.131:0/3639696173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d80b4140 msgr2=0x7fa0d80d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:07.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.134+0000 7fa0cde00640 1 --2- 172.21.15.131:0/3639696173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d80b4140 0x7fa0d80d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.134+0000 7fa0cde00640 1 -- 172.21.15.131:0/3639696173 >> v2:172.21.15.157:3300/0 conn(0x7fa0d81467c0 msgr2=0x7fa0d8072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:07.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.134+0000 7fa0cde00640 1 --2- 172.21.15.131:0/3639696173 >> v2:172.21.15.157:3300/0 conn(0x7fa0d81467c0 0x7fa0d8072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.134+0000 7fa0cde00640 1 -- 172.21.15.131:0/3639696173 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d80734a0 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.134+0000 7fa0d6800640 1 --2- 172.21.15.131:0/3639696173 >> v2:172.21.15.157:3300/0 conn(0x7fa0d81467c0 0x7fa0d8072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:07.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.135+0000 7fa0cde00640 1 --2- 172.21.15.131:0/3639696173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0d80b3ce0 0x7fa0d80d0b20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa0d806d150 tx=0x7fa0c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93505884b2d1a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:07.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.136+0000 7fa0d5e00640 1 -- 172.21.15.131:0/3639696173 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0037070 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.136+0000 7fa0d5e00640 1 -- 172.21.15.131:0/3639696173 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c0032440 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.136+0000 7fa0d5e00640 1 -- 172.21.15.131:0/3639696173 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0032810 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.136+0000 7fa0dd400640 1 -- 172.21.15.131:0/3639696173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0d80b3ce0 msgr2=0x7fa0d80d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.136+0000 7fa0dd400640 1 --2- 172.21.15.131:0/3639696173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0d80b3ce0 0x7fa0d80d0b20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa0d806d150 tx=0x7fa0c002a1e0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.137+0000 7fa0dd400640 1 -- 172.21.15.131:0/3639696173 shutdown_connections 2024-04-28T12:43:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.137+0000 7fa0dd400640 1 --2- 172.21.15.131:0/3639696173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0d80b3ce0 0x7fa0d80d0b20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.137+0000 7fa0dd400640 1 --2- 172.21.15.131:0/3639696173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d80b4140 0x7fa0d80d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.137+0000 7fa0dd400640 1 --2- 172.21.15.131:0/3639696173 >> v2:172.21.15.157:3300/0 conn(0x7fa0d81467c0 0x7fa0d8072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.137+0000 7fa0dd400640 1 -- 172.21.15.131:0/3639696173 >> 172.21.15.131:0/3639696173 conn(0x7fa0d80b2900 msgr2=0x7fa0d80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:07.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.137+0000 7fa0dd400640 1 -- 172.21.15.131:0/3639696173 shutdown_connections 2024-04-28T12:43:07.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.137+0000 7fa0dd400640 1 -- 172.21.15.131:0/3639696173 wait complete. 2024-04-28T12:43:07.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.138+0000 7fa0dd400640 1 Processor -- start 2024-04-28T12:43:07.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.139+0000 7fa0dd400640 1 -- start start 2024-04-28T12:43:07.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.139+0000 7fa0dd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0d80b3ce0 0x7fa0d815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.139+0000 7fa0dd400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d81467c0 0x7fa0d815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.139+0000 7fa0dd400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d815d7e0 0x7fa0d8141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.139+0000 7fa0dd400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8160a20 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.139+0000 7fa0dd400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8073a60 con 0x7fa0d815d7e0 2024-04-28T12:43:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.139+0000 7fa0dd400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d8160ba0 con 0x7fa0d81467c0 2024-04-28T12:43:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.140+0000 7fa0d6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0d80b3ce0 0x7fa0d815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.140+0000 7fa0d6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0d80b3ce0 0x7fa0d815ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53830/0 (socket says 172.21.15.131:53830) 2024-04-28T12:43:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.140+0000 7fa0d6800640 1 -- 172.21.15.131:0/863308286 learned_addr learned my addr 172.21.15.131:0/863308286 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.140+0000 7fa0cde00640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d81467c0 0x7fa0d815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.140+0000 7fa0d7200640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d815d7e0 0x7fa0d8141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.140+0000 7fa0d6800640 1 -- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d81467c0 msgr2=0x7fa0d815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.140+0000 7fa0d6800640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d81467c0 0x7fa0d815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.140+0000 7fa0d6800640 1 -- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d815d7e0 msgr2=0x7fa0d8141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:07.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.140+0000 7fa0d6800640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d815d7e0 0x7fa0d8141950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.140+0000 7fa0d6800640 1 -- 172.21.15.131:0/863308286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d80d0720 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.140+0000 7fa0cde00640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d81467c0 0x7fa0d815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:07.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.140+0000 7fa0d6800640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0d80b3ce0 0x7fa0d815ff20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800c7f0 tx=0x7fa0c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:07.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.141+0000 7fa0cf400640 1 -- 172.21.15.131:0/863308286 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8015070 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.141+0000 7fa0cf400640 1 -- 172.21.15.131:0/863308286 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c8007820 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.141+0000 7fa0cf400640 1 -- 172.21.15.131:0/863308286 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8007bd0 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.141+0000 7fa0dd400640 1 -- 172.21.15.131:0/863308286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d815dd80 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.141+0000 7fa0dd400640 1 -- 172.21.15.131:0/863308286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d815e340 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.142+0000 7fa0dd400640 1 -- 172.21.15.131:0/863308286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d8069350 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.143+0000 7fa0cf400640 1 -- 172.21.15.131:0/863308286 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0c8018070 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.144+0000 7fa0cf400640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa0b406c920 0x7fa0b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.144+0000 7fa0cf400640 1 -- 172.21.15.131:0/863308286 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fa0c808be40 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.145+0000 7fa0cde00640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa0b406c920 0x7fa0b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:07.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.146+0000 7fa0cde00640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa0b406c920 0x7fa0b406ede0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0034000 tx=0x7fa0c002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:07.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.149+0000 7fa0cf400640 1 -- 172.21.15.131:0/863308286 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0d80d0720 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.388+0000 7fa0dd400640 1 -- 172.21.15.131:0/863308286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0d80b22f0 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.389+0000 7fa0cf400640 1 -- 172.21.15.131:0/863308286 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5189 (secure 0 0 0) 0x7fa0c805a020 con 0x7fa0d80b3ce0 2024-04-28T12:43:07.386 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:07.386 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/361318591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":361318591},{"type":"v1","addr":"172.21.15.157:6836","nonce":361318591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/102335005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":102335005},{"type":"v1","addr":"172.21.15.131:6835","nonce":102335005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1609059776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1609059776},{"type":"v1","addr":"172.21.15.131:6833","nonce":1609059776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:39:10.533686+0000","modified":"2024-04-28T12:39:13.595067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34610":{"gid":34610,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/837933884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":837933884},{"type":"v1","addr":"172.21.15.157:6837","nonce":837933884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34610,"qdb_cluster":[34610]},"id":2}]} 2024-04-28T12:43:07.386 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2024-04-28T12:43:07.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.392+0000 7fa0dd400640 1 -- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa0b406c920 msgr2=0x7fa0b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:07.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.392+0000 7fa0dd400640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa0b406c920 0x7fa0b406ede0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0034000 tx=0x7fa0c002e000 comp rx=0 tx=0).stop 2024-04-28T12:43:07.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.392+0000 7fa0dd400640 1 -- 172.21.15.131:0/863308286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0d80b3ce0 msgr2=0x7fa0d815ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:07.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.392+0000 7fa0dd400640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0d80b3ce0 0x7fa0d815ff20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800c7f0 tx=0x7fa0c800ccb0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.392+0000 7fa0dd400640 1 -- 172.21.15.131:0/863308286 shutdown_connections 2024-04-28T12:43:07.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.392+0000 7fa0dd400640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa0b406c920 0x7fa0b406ede0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.392+0000 7fa0dd400640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d815d7e0 0x7fa0d8141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.392+0000 7fa0dd400640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0d81467c0 0x7fa0d815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.392+0000 7fa0dd400640 1 --2- 172.21.15.131:0/863308286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0d80b3ce0 0x7fa0d815ff20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.392+0000 7fa0dd400640 1 -- 172.21.15.131:0/863308286 >> 172.21.15.131:0/863308286 conn(0x7fa0d80b2900 msgr2=0x7fa0d815c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:07.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.392+0000 7fa0dd400640 1 -- 172.21.15.131:0/863308286 shutdown_connections 2024-04-28T12:43:07.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.392+0000 7fa0dd400640 1 -- 172.21.15.131:0/863308286 wait complete. 2024-04-28T12:43:07.407 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:43:07.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.611+0000 7f6634400640 1 Processor -- start 2024-04-28T12:43:07.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.611+0000 7f6634400640 1 -- start start 2024-04-28T12:43:07.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.612+0000 7f6634400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f662c0d1230 0x7f662c167b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.612+0000 7f6634400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f662c146ba0 con 0x7f662c1467c0 2024-04-28T12:43:07.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.612+0000 7f6634400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f662c0731c0 con 0x7f662c072b70 2024-04-28T12:43:07.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.612+0000 7f6634400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f662c073dd0 con 0x7f662c0d1230 2024-04-28T12:43:07.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.613+0000 7f6631a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f662c1467c0 0x7f662c0b3ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54096/0 (socket says 172.21.15.131:54096) 2024-04-28T12:43:07.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.613+0000 7f6631a00640 1 -- 172.21.15.131:0/2343171213 learned_addr learned my addr 172.21.15.131:0/2343171213 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.613+0000 7f6632400640 1 --2- 172.21.15.131:0/2343171213 >> v2:172.21.15.157:3300/0 conn(0x7f662c0d1230 0x7f662c167b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.613+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237902947 0 0) 0x7f662c146ba0 con 0x7f662c1467c0 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.613+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6614003520 con 0x7f662c1467c0 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 404479504 0 0) 0x7f6614003520 con 0x7f662c1467c0 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 >> v2:172.21.15.157:3300/0 conn(0x7f662c0d1230 msgr2=0x7f662c167b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 --2- 172.21.15.131:0/2343171213 >> v2:172.21.15.157:3300/0 conn(0x7f662c0d1230 0x7f662c167b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 >> v1:172.21.15.157:6789/0 conn(0x7f662c072b70 legacy=0x7f662c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662c0734a0 con 0x7f662c1467c0 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f661c001670 con 0x7f662c1467c0 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 >> v1:172.21.15.131:6789/0 conn(0x7f662c1467c0 legacy=0x7f662c0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 --2- 172.21.15.131:0/2343171213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66140039e0 0x7f6614003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 --2- 172.21.15.131:0/2343171213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6614004800 0x7f6614004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 --2- 172.21.15.131:0/2343171213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66140051c0 0x7f661400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f662c073dd0 con 0x7f66140039e0 2024-04-28T12:43:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f661400c090 con 0x7f6614004800 2024-04-28T12:43:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f661400c290 con 0x7f66140051c0 2024-04-28T12:43:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f6631a00640 1 --2- 172.21.15.131:0/2343171213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66140039e0 0x7f6614003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f6631a00640 1 -- 172.21.15.131:0/2343171213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66140051c0 msgr2=0x7f661400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f6631a00640 1 --2- 172.21.15.131:0/2343171213 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66140051c0 0x7f661400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f6631a00640 1 -- 172.21.15.131:0/2343171213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6614004800 msgr2=0x7f6614004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f6631a00640 1 --2- 172.21.15.131:0/2343171213 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6614004800 0x7f6614004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.614+0000 7f6631a00640 1 -- 172.21.15.131:0/2343171213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6614003520 con 0x7f66140039e0 2024-04-28T12:43:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.615+0000 7f6632400640 1 -- 172.21.15.131:0/2343171213 reap_dead start 2024-04-28T12:43:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.615+0000 7f6631a00640 1 --2- 172.21.15.131:0/2343171213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66140039e0 0x7f6614003e00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f661c00c190 tx=0x7f6614005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e60e2e95c230bf server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.616+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c007ce0 con 0x7f66140039e0 2024-04-28T12:43:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.616+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f661c005ad0 con 0x7f66140039e0 2024-04-28T12:43:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.616+0000 7f662be00640 1 -- 172.21.15.131:0/2343171213 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c007070 con 0x7f66140039e0 2024-04-28T12:43:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.616+0000 7f6634400640 1 -- 172.21.15.131:0/2343171213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66140039e0 msgr2=0x7f6614003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.616+0000 7f6634400640 1 --2- 172.21.15.131:0/2343171213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66140039e0 0x7f6614003e00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f661c00c190 tx=0x7f6614005830 comp rx=0 tx=0).stop 2024-04-28T12:43:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.616+0000 7f6634400640 1 -- 172.21.15.131:0/2343171213 shutdown_connections 2024-04-28T12:43:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.616+0000 7f6634400640 1 --2- 172.21.15.131:0/2343171213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66140039e0 0x7f6614003e00 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.616+0000 7f6634400640 1 -- 172.21.15.131:0/2343171213 >> 172.21.15.131:0/2343171213 conn(0x7f662c0b2900 msgr2=0x7f662c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.616+0000 7f6634400640 1 -- 172.21.15.131:0/2343171213 shutdown_connections 2024-04-28T12:43:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.617+0000 7f6634400640 1 -- 172.21.15.131:0/2343171213 wait complete. 2024-04-28T12:43:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.618+0000 7f6634400640 1 Processor -- start 2024-04-28T12:43:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.618+0000 7f6634400640 1 -- start start 2024-04-28T12:43:07.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.619+0000 7f6634400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66140039e0 0x7f662c136ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.619+0000 7f6634400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f662c15c790 0x7f662c15df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.619+0000 7f6634400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c15e4d0 0x7f662c13ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.619+0000 7f6634400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f662c0d0b20 con 0x7f662c15e4d0 2024-04-28T12:43:07.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.619+0000 7f6634400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f662c073a60 con 0x7f66140039e0 2024-04-28T12:43:07.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.619+0000 7f6634400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f662c0d0ca0 con 0x7f662c15c790 2024-04-28T12:43:07.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.620+0000 7f6632400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c15e4d0 0x7f662c13ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:07.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.620+0000 7f6632400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c15e4d0 0x7f662c13ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53848/0 (socket says 172.21.15.131:53848) 2024-04-28T12:43:07.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.620+0000 7f6632400640 1 -- 172.21.15.131:0/2292985489 learned_addr learned my addr 172.21.15.131:0/2292985489 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:07.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.620+0000 7f6632400640 1 -- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f662c15c790 msgr2=0x7f662c15df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:07.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.620+0000 7f6631000640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f662c15c790 0x7f662c15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.620+0000 7f6631a00640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66140039e0 0x7f662c136ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.620+0000 7f6632400640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f662c15c790 0x7f662c15df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.620+0000 7f6632400640 1 -- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66140039e0 msgr2=0x7f662c136ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.620+0000 7f6632400640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66140039e0 0x7f662c136ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.620+0000 7f6632400640 1 -- 172.21.15.131:0/2292985489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662c0ce920 con 0x7f662c15e4d0 2024-04-28T12:43:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.620+0000 7f6632400640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c15e4d0 0x7f662c13ecb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f662c0ce6e0 tx=0x7f662400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.621+0000 7f6629600640 1 -- 172.21.15.131:0/2292985489 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662400c620 con 0x7f662c15e4d0 2024-04-28T12:43:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.621+0000 7f6629600640 1 -- 172.21.15.131:0/2292985489 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f662401a070 con 0x7f662c15e4d0 2024-04-28T12:43:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.621+0000 7f6634400640 1 -- 172.21.15.131:0/2292985489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f662c0cec00 con 0x7f662c15e4d0 2024-04-28T12:43:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.621+0000 7f6629600640 1 -- 172.21.15.131:0/2292985489 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662400ca00 con 0x7f662c15e4d0 2024-04-28T12:43:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.621+0000 7f6634400640 1 -- 172.21.15.131:0/2292985489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f662c0cf1c0 con 0x7f662c15e4d0 2024-04-28T12:43:07.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.622+0000 7f6634400640 1 -- 172.21.15.131:0/2292985489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f662c069350 con 0x7f662c15e4d0 2024-04-28T12:43:07.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.623+0000 7f6629600640 1 -- 172.21.15.131:0/2292985489 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f662400cba0 con 0x7f662c15e4d0 2024-04-28T12:43:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.624+0000 7f6629600640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f661006c920 0x7f661006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:07.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.625+0000 7f6629600640 1 -- 172.21.15.131:0/2292985489 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f662408ba80 con 0x7f662c15e4d0 2024-04-28T12:43:07.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.625+0000 7f6631a00640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f661006c920 0x7f661006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:07.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.626+0000 7f6631a00640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f661006c920 0x7f661006ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f661c007fd0 tx=0x7f661c02d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:07.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.629+0000 7f6629600640 1 -- 172.21.15.131:0/2292985489 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f662408de20 con 0x7f662c15e4d0 2024-04-28T12:43:07.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.831+0000 7f6634400640 1 -- 172.21.15.131:0/2292985489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f662c0cf390 con 0x7f662c15e4d0 2024-04-28T12:43:07.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.834+0000 7f6629600640 1 -- 172.21.15.131:0/2292985489 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16958 (secure 0 0 0) 0x7f662401c310 con 0x7f662c15e4d0 2024-04-28T12:43:07.830 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:07.830 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:40:08.292981+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:39:08.310817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-28T12:39:09.505586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:43:07.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.836+0000 7f6634400640 1 -- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f661006c920 msgr2=0x7f661006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:07.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.836+0000 7f6634400640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f661006c920 0x7f661006ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f661c007fd0 tx=0x7f661c02d000 comp rx=0 tx=0).stop 2024-04-28T12:43:07.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.836+0000 7f6634400640 1 -- 172.21.15.131:0/2292985489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c15e4d0 msgr2=0x7f662c13ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:07.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.836+0000 7f6634400640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c15e4d0 0x7f662c13ecb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f662c0ce6e0 tx=0x7f662400ec00 comp rx=0 tx=0).stop 2024-04-28T12:43:07.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.837+0000 7f6634400640 1 -- 172.21.15.131:0/2292985489 shutdown_connections 2024-04-28T12:43:07.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.837+0000 7f6634400640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f661006c920 0x7f661006ede0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.837+0000 7f6634400640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f662c15e4d0 0x7f662c13ecb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.837+0000 7f6634400640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f662c15c790 0x7f662c15df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.837+0000 7f6634400640 1 --2- 172.21.15.131:0/2292985489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f66140039e0 0x7f662c136ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:07.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.837+0000 7f6634400640 1 -- 172.21.15.131:0/2292985489 >> 172.21.15.131:0/2292985489 conn(0x7f662c0b2900 msgr2=0x7f662c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:07.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.837+0000 7f6634400640 1 -- 172.21.15.131:0/2292985489 shutdown_connections 2024-04-28T12:43:07.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:07.837+0000 7f6634400640 1 -- 172.21.15.131:0/2292985489 wait complete. 2024-04-28T12:43:07.853 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:37:14.943463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:39:08.310817+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:39:09.505586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T12:43:07.853 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:43:08.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.064+0000 7fcc03400640 1 Processor -- start 2024-04-28T12:43:08.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.065+0000 7fcc03400640 1 -- start start 2024-04-28T12:43:08.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.066+0000 7fcc03400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbfc15f1e0 0x7fcbfc06e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.066+0000 7fcc03400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcbfc071ee0 0x7fcbfc072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.067+0000 7fcc03400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbfc06e7b0 con 0x7fcbfc0b57e0 2024-04-28T12:43:08.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.067+0000 7fcc03400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcbfc15f5c0 con 0x7fcbfc071ee0 2024-04-28T12:43:08.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.067+0000 7fcc03400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcbfc0729a0 con 0x7fcbfc15f1e0 2024-04-28T12:43:08.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.067+0000 7fcbf3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcbfc071ee0 0x7fcbfc072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.067+0000 7fcbf3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcbfc071ee0 0x7fcbfc072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53856/0 (socket says 172.21.15.131:53856) 2024-04-28T12:43:08.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.067+0000 7fcbf3e00640 1 -- 172.21.15.131:0/1628994324 learned_addr learned my addr 172.21.15.131:0/1628994324 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:08.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.067+0000 7fcc00a00640 1 --2- 172.21.15.131:0/1628994324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbfc15f1e0 0x7fcbfc06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.067+0000 7fcbf3e00640 1 -- 172.21.15.131:0/1628994324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbfc15f1e0 msgr2=0x7fcbfc06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:08.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.067+0000 7fcbf3e00640 1 --2- 172.21.15.131:0/1628994324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbfc15f1e0 0x7fcbfc06e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.067+0000 7fcbf3e00640 1 -- 172.21.15.131:0/1628994324 >> v1:172.21.15.157:6789/0 conn(0x7fcbfc0b57e0 legacy=0x7fcbfc0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.067+0000 7fcbf3e00640 1 -- 172.21.15.131:0/1628994324 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc072c80 con 0x7fcbfc071ee0 2024-04-28T12:43:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.069+0000 7fcbf3e00640 1 --2- 172.21.15.131:0/1628994324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbfc071ee0 0x7fcbfc072340 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009870 tx=0x7fcbec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54eca0b9a2a6bfd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.069+0000 7fcbfbe00640 1 -- 172.21.15.131:0/1628994324 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec037070 con 0x7fcbfc071ee0 2024-04-28T12:43:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.069+0000 7fcbfbe00640 1 -- 172.21.15.131:0/1628994324 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbec032440 con 0x7fcbfc071ee0 2024-04-28T12:43:08.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.069+0000 7fcbfbe00640 1 -- 172.21.15.131:0/1628994324 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec032810 con 0x7fcbfc071ee0 2024-04-28T12:43:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.070+0000 7fcc03400640 1 -- 172.21.15.131:0/1628994324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbfc071ee0 msgr2=0x7fcbfc072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.070+0000 7fcc03400640 1 --2- 172.21.15.131:0/1628994324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbfc071ee0 0x7fcbfc072340 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009870 tx=0x7fcbec02a210 comp rx=0 tx=0).stop 2024-04-28T12:43:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.070+0000 7fcc03400640 1 -- 172.21.15.131:0/1628994324 shutdown_connections 2024-04-28T12:43:08.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.070+0000 7fcc03400640 1 --2- 172.21.15.131:0/1628994324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbfc071ee0 0x7fcbfc072340 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.070+0000 7fcc03400640 1 --2- 172.21.15.131:0/1628994324 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbfc15f1e0 0x7fcbfc06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.070+0000 7fcc03400640 1 -- 172.21.15.131:0/1628994324 >> 172.21.15.131:0/1628994324 conn(0x7fcbfc0b2900 msgr2=0x7fcbfc0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:08.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.070+0000 7fcc03400640 1 -- 172.21.15.131:0/1628994324 shutdown_connections 2024-04-28T12:43:08.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.070+0000 7fcc03400640 1 -- 172.21.15.131:0/1628994324 wait complete. 2024-04-28T12:43:08.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.072+0000 7fcc03400640 1 Processor -- start 2024-04-28T12:43:08.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.072+0000 7fcc03400640 1 -- start start 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc03400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbfc071ee0 0x7fcbfc14e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc03400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0b57e0 0x7fcbfc15b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc03400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbfc15f1e0 0x7fcbfc15b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc03400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc15f9d0 con 0x7fcbfc071ee0 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc03400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc0b42c0 con 0x7fcbfc0b57e0 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc03400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcbfc0b3f70 con 0x7fcbfc15f1e0 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc00a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbfc071ee0 0x7fcbfc14e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc00a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbfc071ee0 0x7fcbfc14e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53872/0 (socket says 172.21.15.131:53872) 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc00a00640 1 -- 172.21.15.131:0/1563887368 learned_addr learned my addr 172.21.15.131:0/1563887368 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc00a00640 1 -- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbfc15f1e0 msgr2=0x7fcbfc15b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcbf3e00640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0b57e0 0x7fcbfc15b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc01400640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbfc15f1e0 0x7fcbfc15b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc00a00640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbfc15f1e0 0x7fcbfc15b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc00a00640 1 -- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0b57e0 msgr2=0x7fcbfc15b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc00a00640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0b57e0 0x7fcbfc15b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcc00a00640 1 -- 172.21.15.131:0/1563887368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc0ccfb0 con 0x7fcbfc071ee0 2024-04-28T12:43:08.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.073+0000 7fcbf3e00640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0b57e0 0x7fcbfc15b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:08.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.074+0000 7fcc00a00640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbfc071ee0 0x7fcbfc14e870 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400d800 tx=0x7fcbe400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:08.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.074+0000 7fcbf9600640 1 -- 172.21.15.131:0/1563887368 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400b6c0 con 0x7fcbfc071ee0 2024-04-28T12:43:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.074+0000 7fcbf9600640 1 -- 172.21.15.131:0/1563887368 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe4019070 con 0x7fcbfc071ee0 2024-04-28T12:43:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.074+0000 7fcbf9600640 1 -- 172.21.15.131:0/1563887368 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400ba80 con 0x7fcbfc071ee0 2024-04-28T12:43:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.074+0000 7fcc03400640 1 -- 172.21.15.131:0/1563887368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbfc15a2b0 con 0x7fcbfc071ee0 2024-04-28T12:43:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.074+0000 7fcc03400640 1 -- 172.21.15.131:0/1563887368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbfc15ab50 con 0x7fcbfc071ee0 2024-04-28T12:43:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.075+0000 7fcc03400640 1 -- 172.21.15.131:0/1563887368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbfc069350 con 0x7fcbfc071ee0 2024-04-28T12:43:08.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.076+0000 7fcbf9600640 1 -- 172.21.15.131:0/1563887368 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbe400bc20 con 0x7fcbfc071ee0 2024-04-28T12:43:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.077+0000 7fcbf9600640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcbd806c8f0 0x7fcbd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.077+0000 7fcbf9600640 1 -- 172.21.15.131:0/1563887368 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fcbe408c320 con 0x7fcbfc071ee0 2024-04-28T12:43:08.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.078+0000 7fcbf3e00640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcbd806c8f0 0x7fcbd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.079+0000 7fcbf3e00640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcbd806c8f0 0x7fcbd806edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcbec034000 tx=0x7fcbec02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:08.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.082+0000 7fcbf9600640 1 -- 172.21.15.131:0/1563887368 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcbe401c360 con 0x7fcbfc071ee0 2024-04-28T12:43:08.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.280+0000 7fcc03400640 1 -- 172.21.15.131:0/1563887368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcbfc06c780 con 0x7fcbfc071ee0 2024-04-28T12:43:08.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.281+0000 7fcbf9600640 1 -- 172.21.15.131:0/1563887368 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7fcbe405a470 con 0x7fcbfc071ee0 2024-04-28T12:43:08.278 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-04-28T12:43:08.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.284+0000 7fcc03400640 1 -- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcbd806c8f0 msgr2=0x7fcbd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.284+0000 7fcc03400640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcbd806c8f0 0x7fcbd806edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcbec034000 tx=0x7fcbec02e000 comp rx=0 tx=0).stop 2024-04-28T12:43:08.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.284+0000 7fcc03400640 1 -- 172.21.15.131:0/1563887368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbfc071ee0 msgr2=0x7fcbfc14e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.284+0000 7fcc03400640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbfc071ee0 0x7fcbfc14e870 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400d800 tx=0x7fcbe400dcc0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.284+0000 7fcc03400640 1 -- 172.21.15.131:0/1563887368 shutdown_connections 2024-04-28T12:43:08.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.284+0000 7fcc03400640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fcbd806c8f0 0x7fcbd806edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.284+0000 7fcc03400640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcbfc15f1e0 0x7fcbfc15b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.284+0000 7fcc03400640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcbfc0b57e0 0x7fcbfc15b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.284+0000 7fcc03400640 1 --2- 172.21.15.131:0/1563887368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbfc071ee0 0x7fcbfc14e870 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.284+0000 7fcc03400640 1 -- 172.21.15.131:0/1563887368 >> 172.21.15.131:0/1563887368 conn(0x7fcbfc0b2900 msgr2=0x7fcbfc0b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:08.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.284+0000 7fcc03400640 1 -- 172.21.15.131:0/1563887368 shutdown_connections 2024-04-28T12:43:08.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.284+0000 7fcc03400640 1 -- 172.21.15.131:0/1563887368 wait complete. 2024-04-28T12:43:08.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:43:08.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.498+0000 7ffa93200640 1 Processor -- start 2024-04-28T12:43:08.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.498+0000 7ffa93200640 1 -- start start 2024-04-28T12:43:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.500+0000 7ffa93200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7ffa94155cd0 0x7ffa94072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.500+0000 7ffa93200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffa940b5b10 0x7ffa94162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.500+0000 7ffa93200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa9415f920 con 0x7ffa94072b40 2024-04-28T12:43:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.500+0000 7ffa93200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ffa9415fc00 con 0x7ffa940b5b10 2024-04-28T12:43:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.500+0000 7ffa93200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7ffa9415fee0 con 0x7ffa94155cd0 2024-04-28T12:43:08.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.500+0000 7ffa92800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffa940b5b10 0x7ffa94162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.500+0000 7ffa8be00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ffa94072b40 0x7ffa940b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54106/0 (socket says 172.21.15.131:54106) 2024-04-28T12:43:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.500+0000 7ffa8be00640 1 -- 172.21.15.131:0/4052193881 learned_addr learned my addr 172.21.15.131:0/4052193881 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.500+0000 7ffa91e00640 1 --2- 172.21.15.131:0/4052193881 >> v2:172.21.15.157:3300/0 conn(0x7ffa94155cd0 0x7ffa94072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.501+0000 7ffa92800640 1 -- 172.21.15.131:0/4052193881 >> v2:172.21.15.157:3300/0 conn(0x7ffa94155cd0 msgr2=0x7ffa94072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.501+0000 7ffa92800640 1 --2- 172.21.15.131:0/4052193881 >> v2:172.21.15.157:3300/0 conn(0x7ffa94155cd0 0x7ffa94072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.501+0000 7ffa92800640 1 -- 172.21.15.131:0/4052193881 >> v1:172.21.15.131:6789/0 conn(0x7ffa94072b40 legacy=0x7ffa940b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.501+0000 7ffa92800640 1 -- 172.21.15.131:0/4052193881 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa941601c0 con 0x7ffa940b5b10 2024-04-28T12:43:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.501+0000 7ffa91400640 1 -- 172.21.15.131:0/4052193881 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1927302338 0 0) 0x7ffa9415f920 con 0x7ffa94072b40 2024-04-28T12:43:08.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.502+0000 7ffa92800640 1 --2- 172.21.15.131:0/4052193881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa940b5b10 0x7ffa94162f00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ffa940d1320 tx=0x7ffa8c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e0235b50e522b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:08.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.502+0000 7ffa91400640 1 -- 172.21.15.131:0/4052193881 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c007740 con 0x7ffa940b5b10 2024-04-28T12:43:08.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.502+0000 7ffa91400640 1 -- 172.21.15.131:0/4052193881 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa8c0078e0 con 0x7ffa940b5b10 2024-04-28T12:43:08.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.502+0000 7ffa91400640 1 -- 172.21.15.131:0/4052193881 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c007cb0 con 0x7ffa940b5b10 2024-04-28T12:43:08.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.502+0000 7ffa93200640 1 -- 172.21.15.131:0/4052193881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa940b5b10 msgr2=0x7ffa94162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.503+0000 7ffa93200640 1 --2- 172.21.15.131:0/4052193881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa940b5b10 0x7ffa94162f00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ffa940d1320 tx=0x7ffa8c02a710 comp rx=0 tx=0).stop 2024-04-28T12:43:08.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.503+0000 7ffa93200640 1 -- 172.21.15.131:0/4052193881 shutdown_connections 2024-04-28T12:43:08.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.503+0000 7ffa93200640 1 --2- 172.21.15.131:0/4052193881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa940b5b10 0x7ffa94162f00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.503+0000 7ffa93200640 1 --2- 172.21.15.131:0/4052193881 >> v2:172.21.15.157:3300/0 conn(0x7ffa94155cd0 0x7ffa94072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.503+0000 7ffa93200640 1 -- 172.21.15.131:0/4052193881 >> 172.21.15.131:0/4052193881 conn(0x7ffa94068090 msgr2=0x7ffa940684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:08.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.503+0000 7ffa93200640 1 -- 172.21.15.131:0/4052193881 shutdown_connections 2024-04-28T12:43:08.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.503+0000 7ffa93200640 1 -- 172.21.15.131:0/4052193881 wait complete. 2024-04-28T12:43:08.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.504+0000 7ffa93200640 1 Processor -- start 2024-04-28T12:43:08.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.505+0000 7ffa93200640 1 -- start start 2024-04-28T12:43:08.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.505+0000 7ffa93200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa94072b40 0x7ffa941496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa93200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa940b5b10 0x7ffa94153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa93200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa94155cd0 0x7ffa941540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa93200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffa94156230 con 0x7ffa940b5b10 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa93200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941560b0 con 0x7ffa94072b40 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa93200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffa9415f3f0 con 0x7ffa94155cd0 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa8be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa940b5b10 0x7ffa94153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa8be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa940b5b10 0x7ffa94153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53886/0 (socket says 172.21.15.131:53886) 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa8be00640 1 -- 172.21.15.131:0/10532459 learned_addr learned my addr 172.21.15.131:0/10532459 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa91e00640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa94072b40 0x7ffa941496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa92800640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa94155cd0 0x7ffa941540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa8be00640 1 -- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa94155cd0 msgr2=0x7ffa941540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa8be00640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa94155cd0 0x7ffa941540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa8be00640 1 -- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa94072b40 msgr2=0x7ffa941496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa8be00640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa94072b40 0x7ffa941496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa8be00640 1 -- 172.21.15.131:0/10532459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa940b4ca0 con 0x7ffa940b5b10 2024-04-28T12:43:08.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.506+0000 7ffa91e00640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa94072b40 0x7ffa941496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:08.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.507+0000 7ffa8be00640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa940b5b10 0x7ffa94153b70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00c4d0 tx=0x7ffa7c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:08.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.507+0000 7ffa8a000640 1 -- 172.21.15.131:0/10532459 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c018070 con 0x7ffa940b5b10 2024-04-28T12:43:08.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.507+0000 7ffa8a000640 1 -- 172.21.15.131:0/10532459 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c007500 con 0x7ffa940b5b10 2024-04-28T12:43:08.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.507+0000 7ffa93200640 1 -- 172.21.15.131:0/10532459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa940b4f80 con 0x7ffa940b5b10 2024-04-28T12:43:08.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.507+0000 7ffa8a000640 1 -- 172.21.15.131:0/10532459 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c004a20 con 0x7ffa940b5b10 2024-04-28T12:43:08.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.507+0000 7ffa93200640 1 -- 172.21.15.131:0/10532459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa940b5540 con 0x7ffa940b5b10 2024-04-28T12:43:08.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.508+0000 7ffa93200640 1 -- 172.21.15.131:0/10532459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa940506c0 con 0x7ffa940b5b10 2024-04-28T12:43:08.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.509+0000 7ffa8a000640 1 -- 172.21.15.131:0/10532459 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa7c01b070 con 0x7ffa940b5b10 2024-04-28T12:43:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.510+0000 7ffa8a000640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffa7006c8f0 0x7ffa7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.510+0000 7ffa8a000640 1 -- 172.21.15.131:0/10532459 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7ffa7c08b0e0 con 0x7ffa940b5b10 2024-04-28T12:43:08.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.511+0000 7ffa91e00640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffa7006c8f0 0x7ffa7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.512+0000 7ffa91e00640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffa7006c8f0 0x7ffa7006edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffa84006fd0 tx=0x7ffa84008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:08.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.515+0000 7ffa8a000640 1 -- 172.21.15.131:0/10532459 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa7c0592c0 con 0x7ffa940b5b10 2024-04-28T12:43:08.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.718+0000 7ffa93200640 1 -- 172.21.15.131:0/10532459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffa94068840 con 0x7ffa940b5b10 2024-04-28T12:43:08.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.719+0000 7ffa8a000640 1 -- 172.21.15.131:0/10532459 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7ffa7c058bc0 con 0x7ffa940b5b10 2024-04-28T12:43:08.716 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-04-28T12:43:08.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.722+0000 7ffa93200640 1 -- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffa7006c8f0 msgr2=0x7ffa7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.722+0000 7ffa93200640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffa7006c8f0 0x7ffa7006edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffa84006fd0 tx=0x7ffa84008040 comp rx=0 tx=0).stop 2024-04-28T12:43:08.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.722+0000 7ffa93200640 1 -- 172.21.15.131:0/10532459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa940b5b10 msgr2=0x7ffa94153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.722+0000 7ffa93200640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa940b5b10 0x7ffa94153b70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00c4d0 tx=0x7ffa7c00c990 comp rx=0 tx=0).stop 2024-04-28T12:43:08.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.722+0000 7ffa93200640 1 -- 172.21.15.131:0/10532459 shutdown_connections 2024-04-28T12:43:08.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.722+0000 7ffa93200640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7ffa7006c8f0 0x7ffa7006edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.722+0000 7ffa93200640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa94155cd0 0x7ffa941540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.722+0000 7ffa93200640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa940b5b10 0x7ffa94153b70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.722+0000 7ffa93200640 1 --2- 172.21.15.131:0/10532459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa94072b40 0x7ffa941496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.722+0000 7ffa93200640 1 -- 172.21.15.131:0/10532459 >> 172.21.15.131:0/10532459 conn(0x7ffa94068090 msgr2=0x7ffa940b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:08.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.722+0000 7ffa93200640 1 -- 172.21.15.131:0/10532459 shutdown_connections 2024-04-28T12:43:08.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.723+0000 7ffa93200640 1 -- 172.21.15.131:0/10532459 wait complete. 2024-04-28T12:43:08.740 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:43:08.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.949+0000 7eff87600640 1 Processor -- start 2024-04-28T12:43:08.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.949+0000 7eff87600640 1 -- start start 2024-04-28T12:43:08.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.951+0000 7eff87600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7eff80159870 0x7eff8006efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.951+0000 7eff87600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff80073740 0x7eff80073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.951+0000 7eff87600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff800b48f0 con 0x7eff800b3c00 2024-04-28T12:43:08.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.951+0000 7eff87600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7eff800740e0 con 0x7eff80159870 2024-04-28T12:43:08.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.951+0000 7eff87600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff80159c50 con 0x7eff80073740 2024-04-28T12:43:08.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.951+0000 7eff84c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7eff80159870 0x7eff8006efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.951+0000 7eff84c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7eff80159870 0x7eff8006efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53892/0 (socket says 172.21.15.131:53892) 2024-04-28T12:43:08.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.951+0000 7eff84c00640 1 -- 172.21.15.131:0/313071988 learned_addr learned my addr 172.21.15.131:0/313071988 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:08.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.951+0000 7eff77e00640 1 --2- 172.21.15.131:0/313071988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff80073740 0x7eff80073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.951+0000 7eff84c00640 1 -- 172.21.15.131:0/313071988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff80073740 msgr2=0x7eff80073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:08.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.952+0000 7eff84c00640 1 --2- 172.21.15.131:0/313071988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff80073740 0x7eff80073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.952+0000 7eff84c00640 1 -- 172.21.15.131:0/313071988 >> v1:172.21.15.157:6789/0 conn(0x7eff800b3c00 legacy=0x7eff800b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:08.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.952+0000 7eff84c00640 1 -- 172.21.15.131:0/313071988 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff800b4bd0 con 0x7eff80159870 2024-04-28T12:43:08.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.952+0000 7eff77e00640 1 --2- 172.21.15.131:0/313071988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff80073740 0x7eff80073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:08.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.953+0000 7eff84c00640 1 --2- 172.21.15.131:0/313071988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff80159870 0x7eff8006efa0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7eff70009870 tx=0x7eff7002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb8af4daf19b0ca server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:08.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.953+0000 7eff7fe00640 1 -- 172.21.15.131:0/313071988 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70037070 con 0x7eff80159870 2024-04-28T12:43:08.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.953+0000 7eff7fe00640 1 -- 172.21.15.131:0/313071988 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff70032440 con 0x7eff80159870 2024-04-28T12:43:08.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.953+0000 7eff7fe00640 1 -- 172.21.15.131:0/313071988 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70032810 con 0x7eff80159870 2024-04-28T12:43:08.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.953+0000 7eff87600640 1 -- 172.21.15.131:0/313071988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff80159870 msgr2=0x7eff8006efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.953+0000 7eff87600640 1 --2- 172.21.15.131:0/313071988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff80159870 0x7eff8006efa0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7eff70009870 tx=0x7eff7002a200 comp rx=0 tx=0).stop 2024-04-28T12:43:08.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.954+0000 7eff87600640 1 -- 172.21.15.131:0/313071988 shutdown_connections 2024-04-28T12:43:08.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.954+0000 7eff87600640 1 --2- 172.21.15.131:0/313071988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff80073740 0x7eff80073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.954+0000 7eff87600640 1 --2- 172.21.15.131:0/313071988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff80159870 0x7eff8006efa0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.954+0000 7eff87600640 1 -- 172.21.15.131:0/313071988 >> 172.21.15.131:0/313071988 conn(0x7eff800b2900 msgr2=0x7eff800b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:08.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.954+0000 7eff87600640 1 -- 172.21.15.131:0/313071988 shutdown_connections 2024-04-28T12:43:08.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.954+0000 7eff87600640 1 -- 172.21.15.131:0/313071988 wait complete. 2024-04-28T12:43:08.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.955+0000 7eff87600640 1 Processor -- start 2024-04-28T12:43:08.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.956+0000 7eff87600640 1 -- start start 2024-04-28T12:43:08.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff87600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff80073740 0x7eff800d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff87600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff800b3c00 0x7eff80156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff87600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff80159870 0x7eff8015d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff87600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7eff8006f4e0 con 0x7eff80159870 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff87600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff8006e7a0 con 0x7eff80073740 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff87600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff80071f60 con 0x7eff800b3c00 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff85600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff80159870 0x7eff8015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff85600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff80159870 0x7eff8015d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53894/0 (socket says 172.21.15.131:53894) 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff85600640 1 -- 172.21.15.131:0/3880344803 learned_addr learned my addr 172.21.15.131:0/3880344803 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff84c00640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff80073740 0x7eff800d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff77e00640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff800b3c00 0x7eff80156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff85600640 1 -- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff800b3c00 msgr2=0x7eff80156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff85600640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff800b3c00 0x7eff80156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff85600640 1 -- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff80073740 msgr2=0x7eff800d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff85600640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff80073740 0x7eff800d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff85600640 1 -- 172.21.15.131:0/3880344803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff801558e0 con 0x7eff80159870 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.957+0000 7eff77e00640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff800b3c00 0x7eff80156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.958+0000 7eff85600640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff80159870 0x7eff8015d360 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c7b0 tx=0x7eff7800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:08.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.958+0000 7eff7d600640 1 -- 172.21.15.131:0/3880344803 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78004b60 con 0x7eff80159870 2024-04-28T12:43:08.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.958+0000 7eff7d600640 1 -- 172.21.15.131:0/3880344803 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff78004d00 con 0x7eff80159870 2024-04-28T12:43:08.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.958+0000 7eff7d600640 1 -- 172.21.15.131:0/3880344803 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7800aa30 con 0x7eff80159870 2024-04-28T12:43:08.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.958+0000 7eff87600640 1 -- 172.21.15.131:0/3880344803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff80155bc0 con 0x7eff80159870 2024-04-28T12:43:08.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.958+0000 7eff87600640 1 -- 172.21.15.131:0/3880344803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff80156180 con 0x7eff80159870 2024-04-28T12:43:08.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.959+0000 7eff87600640 1 -- 172.21.15.131:0/3880344803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff80072ad0 con 0x7eff80159870 2024-04-28T12:43:08.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.960+0000 7eff7d600640 1 -- 172.21.15.131:0/3880344803 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff7800abd0 con 0x7eff80159870 2024-04-28T12:43:08.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.961+0000 7eff7d600640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7eff5c06c8f0 0x7eff5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:08.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.961+0000 7eff7d600640 1 -- 172.21.15.131:0/3880344803 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7eff7808bf30 con 0x7eff80159870 2024-04-28T12:43:08.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.962+0000 7eff84c00640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7eff5c06c8f0 0x7eff5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:08.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.963+0000 7eff84c00640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7eff5c06c8f0 0x7eff5c06edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7eff70034000 tx=0x7eff7002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:08.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:08.965+0000 7eff7d600640 1 -- 172.21.15.131:0/3880344803 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff80072ad0 con 0x7eff80159870 2024-04-28T12:43:09.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.167+0000 7eff87600640 1 -- 172.21.15.131:0/3880344803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7eff80069350 con 0x7eff80159870 2024-04-28T12:43:09.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.167+0000 7eff7d600640 1 -- 172.21.15.131:0/3880344803 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7eff78020070 con 0x7eff80159870 2024-04-28T12:43:09.164 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-04-28T12:43:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.170+0000 7eff87600640 1 -- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7eff5c06c8f0 msgr2=0x7eff5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.170+0000 7eff87600640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7eff5c06c8f0 0x7eff5c06edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7eff70034000 tx=0x7eff7002e000 comp rx=0 tx=0).stop 2024-04-28T12:43:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.170+0000 7eff87600640 1 -- 172.21.15.131:0/3880344803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff80159870 msgr2=0x7eff8015d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.170+0000 7eff87600640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff80159870 0x7eff8015d360 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c7b0 tx=0x7eff7800cc70 comp rx=0 tx=0).stop 2024-04-28T12:43:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.170+0000 7eff87600640 1 -- 172.21.15.131:0/3880344803 shutdown_connections 2024-04-28T12:43:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.170+0000 7eff87600640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7eff5c06c8f0 0x7eff5c06edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.170+0000 7eff87600640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff80159870 0x7eff8015d360 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.170+0000 7eff87600640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff800b3c00 0x7eff80156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.170+0000 7eff87600640 1 --2- 172.21.15.131:0/3880344803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff80073740 0x7eff800d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.170+0000 7eff87600640 1 -- 172.21.15.131:0/3880344803 >> 172.21.15.131:0/3880344803 conn(0x7eff800b2900 msgr2=0x7eff800723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.170+0000 7eff87600640 1 -- 172.21.15.131:0/3880344803 shutdown_connections 2024-04-28T12:43:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.171+0000 7eff87600640 1 -- 172.21.15.131:0/3880344803 wait complete. 2024-04-28T12:43:09.186 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:43:09.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.379+0000 7f6715000640 1 Processor -- start 2024-04-28T12:43:09.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.380+0000 7f6715000640 1 -- start start 2024-04-28T12:43:09.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.381+0000 7f6715000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6710159050 0x7f67100b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.381+0000 7f6715000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67100b3c00 0x7f67100b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.381+0000 7f6715000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6710159430 con 0x7f67100d1440 2024-04-28T12:43:09.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.381+0000 7f6715000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f67100b5dd0 con 0x7f6710159050 2024-04-28T12:43:09.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.381+0000 7f6715000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6710073c10 con 0x7f67100b3c00 2024-04-28T12:43:09.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.381+0000 7f670de00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6710159050 0x7f67100b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.382+0000 7f670de00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6710159050 0x7f67100b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53896/0 (socket says 172.21.15.131:53896) 2024-04-28T12:43:09.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.382+0000 7f670de00640 1 -- 172.21.15.131:0/1062195502 learned_addr learned my addr 172.21.15.131:0/1062195502 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:09.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.382+0000 7f670e800640 1 --2- 172.21.15.131:0/1062195502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67100b3c00 0x7f67100b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.382+0000 7f670de00640 1 -- 172.21.15.131:0/1062195502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67100b3c00 msgr2=0x7f67100b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:09.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.382+0000 7f670de00640 1 --2- 172.21.15.131:0/1062195502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67100b3c00 0x7f67100b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.382+0000 7f670de00640 1 -- 172.21.15.131:0/1062195502 >> v1:172.21.15.157:6789/0 conn(0x7f67100d1440 legacy=0x7f671015efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:09.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.382+0000 7f670de00640 1 -- 172.21.15.131:0/1062195502 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6710073480 con 0x7f6710159050 2024-04-28T12:43:09.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.382+0000 7f670e800640 1 --2- 172.21.15.131:0/1062195502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67100b3c00 0x7f67100b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:09.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.383+0000 7f670de00640 1 --2- 172.21.15.131:0/1062195502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6710159050 0x7f67100b5350 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f670000aa10 tx=0x7f670002b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c5047e3c43843e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.383+0000 7f670d400640 1 -- 172.21.15.131:0/1062195502 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670000b040 con 0x7f6710159050 2024-04-28T12:43:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.383+0000 7f670d400640 1 -- 172.21.15.131:0/1062195502 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6700030710 con 0x7f6710159050 2024-04-28T12:43:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.384+0000 7f670d400640 1 -- 172.21.15.131:0/1062195502 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700030ae0 con 0x7f6710159050 2024-04-28T12:43:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.384+0000 7f6715000640 1 -- 172.21.15.131:0/1062195502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6710159050 msgr2=0x7f67100b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.384+0000 7f6715000640 1 --2- 172.21.15.131:0/1062195502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6710159050 0x7f67100b5350 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f670000aa10 tx=0x7f670002b5d0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.384+0000 7f6715000640 1 -- 172.21.15.131:0/1062195502 shutdown_connections 2024-04-28T12:43:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.384+0000 7f6715000640 1 --2- 172.21.15.131:0/1062195502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67100b3c00 0x7f67100b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.384+0000 7f6715000640 1 --2- 172.21.15.131:0/1062195502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6710159050 0x7f67100b5350 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.384+0000 7f6715000640 1 -- 172.21.15.131:0/1062195502 >> 172.21.15.131:0/1062195502 conn(0x7f67100b2900 msgr2=0x7f67100b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.384+0000 7f6715000640 1 -- 172.21.15.131:0/1062195502 shutdown_connections 2024-04-28T12:43:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.384+0000 7f6715000640 1 -- 172.21.15.131:0/1062195502 wait complete. 2024-04-28T12:43:09.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.386+0000 7f6715000640 1 Processor -- start 2024-04-28T12:43:09.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.386+0000 7f6715000640 1 -- start start 2024-04-28T12:43:09.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.387+0000 7f6715000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67100b3c00 0x7f671014daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.387+0000 7f6715000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67100d1440 0x7f67100aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.387+0000 7f6715000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6710159050 0x7f671006e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.387+0000 7f6715000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6710071f50 con 0x7f6710159050 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.387+0000 7f6715000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6710073e60 con 0x7f67100d1440 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.387+0000 7f6715000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6710074140 con 0x7f67100b3c00 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.387+0000 7f670e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6710159050 0x7f671006e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.387+0000 7f670e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6710159050 0x7f671006e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53900/0 (socket says 172.21.15.131:53900) 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.387+0000 7f670e800640 1 -- 172.21.15.131:0/2552810952 learned_addr learned my addr 172.21.15.131:0/2552810952 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.388+0000 7f670e800640 1 -- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67100b3c00 msgr2=0x7f671014daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.388+0000 7f6707e00640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67100d1440 0x7f67100aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.388+0000 7f670de00640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67100b3c00 0x7f671014daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.388+0000 7f670e800640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67100b3c00 0x7f671014daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.388+0000 7f670e800640 1 -- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67100d1440 msgr2=0x7f67100aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:09.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.388+0000 7f670e800640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67100d1440 0x7f67100aa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.388+0000 7f670e800640 1 -- 172.21.15.131:0/2552810952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67100aa9f0 con 0x7f6710159050 2024-04-28T12:43:09.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.388+0000 7f670e800640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6710159050 0x7f671006e1f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f670800cef0 tx=0x7f670800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:09.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.388+0000 7f6706000640 1 -- 172.21.15.131:0/2552810952 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670801a070 con 0x7f6710159050 2024-04-28T12:43:09.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.388+0000 7f6706000640 1 -- 172.21.15.131:0/2552810952 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670800d860 con 0x7f6710159050 2024-04-28T12:43:09.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.388+0000 7f6706000640 1 -- 172.21.15.131:0/2552810952 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670800dc50 con 0x7f6710159050 2024-04-28T12:43:09.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.389+0000 7f6715000640 1 -- 172.21.15.131:0/2552810952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67100aac70 con 0x7f6710159050 2024-04-28T12:43:09.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.389+0000 7f6715000640 1 -- 172.21.15.131:0/2552810952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67100ab230 con 0x7f6710159050 2024-04-28T12:43:09.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.390+0000 7f6715000640 1 -- 172.21.15.131:0/2552810952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6710069350 con 0x7f6710159050 2024-04-28T12:43:09.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.391+0000 7f6706000640 1 -- 172.21.15.131:0/2552810952 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f670801d070 con 0x7f6710159050 2024-04-28T12:43:09.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.392+0000 7f6706000640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f66ec06c8f0 0x7f66ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.392+0000 7f6706000640 1 -- 172.21.15.131:0/2552810952 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f670808c5e0 con 0x7f6710159050 2024-04-28T12:43:09.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.392+0000 7f670de00640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f66ec06c8f0 0x7f66ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.393+0000 7f670de00640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f66ec06c8f0 0x7f66ec06edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6700035000 tx=0x7f6700034040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.396+0000 7f6706000640 1 -- 172.21.15.131:0/2552810952 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f67100aa9f0 con 0x7f6710159050 2024-04-28T12:43:09.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.635+0000 7f6715000640 1 -- 172.21.15.131:0/2552810952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f671015c360 con 0x7f6710159050 2024-04-28T12:43:09.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.636+0000 7f6706000640 1 -- 172.21.15.131:0/2552810952 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5189 (secure 0 0 0) 0x7f670805a910 con 0x7f6710159050 2024-04-28T12:43:09.633 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:09.633 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/361318591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":361318591},{"type":"v1","addr":"172.21.15.157:6836","nonce":361318591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/102335005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":102335005},{"type":"v1","addr":"172.21.15.131:6835","nonce":102335005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1609059776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1609059776},{"type":"v1","addr":"172.21.15.131:6833","nonce":1609059776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:39:10.533686+0000","modified":"2024-04-28T12:39:13.595067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34610":{"gid":34610,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/837933884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":837933884},{"type":"v1","addr":"172.21.15.157:6837","nonce":837933884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34610,"qdb_cluster":[34610]},"id":2}]} 2024-04-28T12:43:09.633 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2024-04-28T12:43:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.639+0000 7f6715000640 1 -- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f66ec06c8f0 msgr2=0x7f66ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.639+0000 7f6715000640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f66ec06c8f0 0x7f66ec06edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6700035000 tx=0x7f6700034040 comp rx=0 tx=0).stop 2024-04-28T12:43:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.639+0000 7f6715000640 1 -- 172.21.15.131:0/2552810952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6710159050 msgr2=0x7f671006e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.639+0000 7f6715000640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6710159050 0x7f671006e1f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f670800cef0 tx=0x7f670800a710 comp rx=0 tx=0).stop 2024-04-28T12:43:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.639+0000 7f6715000640 1 -- 172.21.15.131:0/2552810952 shutdown_connections 2024-04-28T12:43:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.639+0000 7f6715000640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f66ec06c8f0 0x7f66ec06edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.639+0000 7f6715000640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6710159050 0x7f671006e1f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.639+0000 7f6715000640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67100d1440 0x7f67100aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.639+0000 7f6715000640 1 --2- 172.21.15.131:0/2552810952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67100b3c00 0x7f671014daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.639+0000 7f6715000640 1 -- 172.21.15.131:0/2552810952 >> 172.21.15.131:0/2552810952 conn(0x7f67100b2900 msgr2=0x7f67100cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.639+0000 7f6715000640 1 -- 172.21.15.131:0/2552810952 shutdown_connections 2024-04-28T12:43:09.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.639+0000 7f6715000640 1 -- 172.21.15.131:0/2552810952 wait complete. 2024-04-28T12:43:09.654 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:43:09.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.858+0000 7f5b0ac00640 1 Processor -- start 2024-04-28T12:43:09.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.858+0000 7f5b0ac00640 1 -- start start 2024-04-28T12:43:09.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.860+0000 7f5b0ac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b041467c0 0x7f5b04072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.860+0000 7f5b0ac00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5b040b4140 0x7f5b0406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.860+0000 7f5b0ac00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b04146ba0 con 0x7f5b040b3ce0 2024-04-28T12:43:09.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.860+0000 7f5b0ac00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5b04073dd0 con 0x7f5b040b4140 2024-04-28T12:43:09.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.860+0000 7f5b0ac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5b040731c0 con 0x7f5b041467c0 2024-04-28T12:43:09.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.860+0000 7f5b03400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5b040b3ce0 0x7f5b040d0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54120/0 (socket says 172.21.15.131:54120) 2024-04-28T12:43:09.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.860+0000 7f5b03400640 1 -- 172.21.15.131:0/1979284054 learned_addr learned my addr 172.21.15.131:0/1979284054 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b03e00640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b041467c0 0x7f5b04072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b08c00640 1 --2- 172.21.15.131:0/1979284054 >> v2:172.21.15.157:3300/0 conn(0x7f5b040b4140 0x7f5b0406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1538265867 0 0) 0x7f5b04146ba0 con 0x7f5b040b3ce0 2024-04-28T12:43:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ae8003520 con 0x7f5b040b3ce0 2024-04-28T12:43:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1207519642 0 0) 0x7f5ae8003520 con 0x7f5b040b3ce0 2024-04-28T12:43:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b041467c0 msgr2=0x7f5b04072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b041467c0 0x7f5b04072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 >> v2:172.21.15.157:3300/0 conn(0x7f5b040b4140 msgr2=0x7f5b0406dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 --2- 172.21.15.131:0/1979284054 >> v2:172.21.15.157:3300/0 conn(0x7f5b040b4140 0x7f5b0406dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b040734a0 con 0x7f5b040b3ce0 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f5aec001670 con 0x7f5b040b3ce0 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 >> v1:172.21.15.131:6789/0 conn(0x7f5b040b3ce0 legacy=0x7f5b040d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae80039b0 0x7f5ae8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8004800 0x7f5ae8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b041467c0 0x7f5ae80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b04073dd0 con 0x7f5ae8004800 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b040731c0 con 0x7f5ae80039b0 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae800baa0 con 0x7f5b041467c0 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.861+0000 7f5b03400640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8004800 0x7f5ae8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.862+0000 7f5b08c00640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b041467c0 0x7f5ae80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.862+0000 7f5b03400640 1 -- 172.21.15.131:0/1979284054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b041467c0 msgr2=0x7f5ae80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.862+0000 7f5b03e00640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae80039b0 0x7f5ae8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.862+0000 7f5b03400640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b041467c0 0x7f5ae80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.862+0000 7f5b03400640 1 -- 172.21.15.131:0/1979284054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae80039b0 msgr2=0x7f5ae8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.862+0000 7f5b03400640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ae80039b0 0x7f5ae8003df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.862+0000 7f5b03400640 1 -- 172.21.15.131:0/1979284054 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae8003520 con 0x7f5ae8004800 2024-04-28T12:43:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.862+0000 7f5b08c00640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b041467c0 0x7f5ae80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.863+0000 7f5b03400640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8004800 0x7f5ae8004c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5aec007fd0 tx=0x7f5ae8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bee5440e815ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.863+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec007d50 con 0x7f5ae8004800 2024-04-28T12:43:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.863+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aec0058d0 con 0x7f5ae8004800 2024-04-28T12:43:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.863+0000 7f5b02a00640 1 -- 172.21.15.131:0/1979284054 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec005ce0 con 0x7f5ae8004800 2024-04-28T12:43:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.864+0000 7f5b0ac00640 1 -- 172.21.15.131:0/1979284054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8004800 msgr2=0x7f5ae8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.864+0000 7f5b0ac00640 1 --2- 172.21.15.131:0/1979284054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8004800 0x7f5ae8004c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5aec007fd0 tx=0x7f5ae8005830 comp rx=0 tx=0).stop 2024-04-28T12:43:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.864+0000 7f5b08c00640 1 -- 172.21.15.131:0/1979284054 reap_dead start 2024-04-28T12:43:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.864+0000 7f5b0ac00640 1 -- 172.21.15.131:0/1979284054 shutdown_connections 2024-04-28T12:43:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.864+0000 7f5b0ac00640 1 -- 172.21.15.131:0/1979284054 >> 172.21.15.131:0/1979284054 conn(0x7f5b040b2900 msgr2=0x7f5b040b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.864+0000 7f5b0ac00640 1 -- 172.21.15.131:0/1979284054 shutdown_connections 2024-04-28T12:43:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.864+0000 7f5b0ac00640 1 -- 172.21.15.131:0/1979284054 wait complete. 2024-04-28T12:43:09.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.866+0000 7f5b0ac00640 1 Processor -- start 2024-04-28T12:43:09.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.866+0000 7f5b0ac00640 1 -- start start 2024-04-28T12:43:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.867+0000 7f5b0ac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8004800 0x7f5b0415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.867+0000 7f5b0ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b041f8420 0x7f5b0415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.867+0000 7f5b0ac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b04140150 0x7f5b04140600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.867+0000 7f5b0ac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5b040d1230 con 0x7f5ae8004800 2024-04-28T12:43:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.867+0000 7f5b0ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b04073a60 con 0x7f5b041f8420 2024-04-28T12:43:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.867+0000 7f5b0ac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5b040d13b0 con 0x7f5b04140150 2024-04-28T12:43:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.867+0000 7f5b03e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8004800 0x7f5b0415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.867+0000 7f5b03e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8004800 0x7f5b0415c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53918/0 (socket says 172.21.15.131:53918) 2024-04-28T12:43:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.867+0000 7f5b03e00640 1 -- 172.21.15.131:0/2252547769 learned_addr learned my addr 172.21.15.131:0/2252547769 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.867+0000 7f5b08c00640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b04140150 0x7f5b04140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5b03400640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b041f8420 0x7f5b0415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5b03e00640 1 -- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b04140150 msgr2=0x7f5b04140600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5b03e00640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b04140150 0x7f5b04140600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5b03e00640 1 -- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b041f8420 msgr2=0x7f5b0415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5b03e00640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b041f8420 0x7f5b0415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5b03e00640 1 -- 172.21.15.131:0/2252547769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0415e980 con 0x7f5ae8004800 2024-04-28T12:43:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5b03400640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b041f8420 0x7f5b0415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5b03e00640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8004800 0x7f5b0415c910 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5afc007ec0 tx=0x7f5afc00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5afbe00640 1 -- 172.21.15.131:0/2252547769 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc018020 con 0x7f5ae8004800 2024-04-28T12:43:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5afbe00640 1 -- 172.21.15.131:0/2252547769 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc0145f0 con 0x7f5ae8004800 2024-04-28T12:43:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5b0ac00640 1 -- 172.21.15.131:0/2252547769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b041f8b90 con 0x7f5ae8004800 2024-04-28T12:43:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5afbe00640 1 -- 172.21.15.131:0/2252547769 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0149c0 con 0x7f5ae8004800 2024-04-28T12:43:09.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.868+0000 7f5b0ac00640 1 -- 172.21.15.131:0/2252547769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b041f9150 con 0x7f5ae8004800 2024-04-28T12:43:09.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.870+0000 7f5b0ac00640 1 -- 172.21.15.131:0/2252547769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b041f9350 con 0x7f5ae8004800 2024-04-28T12:43:09.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.871+0000 7f5afbe00640 1 -- 172.21.15.131:0/2252547769 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5afc01b020 con 0x7f5ae8004800 2024-04-28T12:43:09.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.871+0000 7f5afbe00640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ae006c920 0x7f5ae006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:09.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.872+0000 7f5afbe00640 1 -- 172.21.15.131:0/2252547769 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f5afc08c6c0 con 0x7f5ae8004800 2024-04-28T12:43:09.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.872+0000 7f5b03400640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ae006c920 0x7f5ae006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:09.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.874+0000 7f5b03400640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ae006c920 0x7f5ae006ede0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5aec033000 tx=0x7f5aec009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:09.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:09.876+0000 7f5afbe00640 1 -- 172.21.15.131:0/2252547769 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5afc022020 con 0x7f5ae8004800 2024-04-28T12:43:10.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.074+0000 7f5b0ac00640 1 -- 172.21.15.131:0/2252547769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b04069350 con 0x7f5ae8004800 2024-04-28T12:43:10.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.077+0000 7f5afbe00640 1 -- 172.21.15.131:0/2252547769 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16958 (secure 0 0 0) 0x7f5afc05a9d0 con 0x7f5ae8004800 2024-04-28T12:43:10.074 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:10.074 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:40:08.292981+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:39:08.310817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-28T12:39:09.505586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:43:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.080+0000 7f5b0ac00640 1 -- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ae006c920 msgr2=0x7f5ae006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.080+0000 7f5b0ac00640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ae006c920 0x7f5ae006ede0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5aec033000 tx=0x7f5aec009030 comp rx=0 tx=0).stop 2024-04-28T12:43:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.080+0000 7f5b0ac00640 1 -- 172.21.15.131:0/2252547769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8004800 msgr2=0x7f5b0415c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.080+0000 7f5b0ac00640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8004800 0x7f5b0415c910 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5afc007ec0 tx=0x7f5afc00f520 comp rx=0 tx=0).stop 2024-04-28T12:43:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.080+0000 7f5b0ac00640 1 -- 172.21.15.131:0/2252547769 shutdown_connections 2024-04-28T12:43:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.080+0000 7f5b0ac00640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5ae006c920 0x7f5ae006ede0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.080+0000 7f5b0ac00640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b04140150 0x7f5b04140600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.080+0000 7f5b0ac00640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b041f8420 0x7f5b0415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.080+0000 7f5b0ac00640 1 --2- 172.21.15.131:0/2252547769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ae8004800 0x7f5b0415c910 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.080+0000 7f5b0ac00640 1 -- 172.21.15.131:0/2252547769 >> 172.21.15.131:0/2252547769 conn(0x7f5b040b2900 msgr2=0x7f5b040ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:10.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.080+0000 7f5b0ac00640 1 -- 172.21.15.131:0/2252547769 shutdown_connections 2024-04-28T12:43:10.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.080+0000 7f5b0ac00640 1 -- 172.21.15.131:0/2252547769 wait complete. 2024-04-28T12:43:10.097 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-28T12:43:10.097 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:43:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.290+0000 7f68b7e00640 1 Processor -- start 2024-04-28T12:43:10.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.290+0000 7f68b7e00640 1 -- start start 2024-04-28T12:43:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.291+0000 7f68b7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68b80cd180 0x7f68b8072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.292+0000 7f68b7e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68b80cd560 con 0x7f68b80ce480 2024-04-28T12:43:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.292+0000 7f68b7e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68b8072d00 con 0x7f68b815dc50 2024-04-28T12:43:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.292+0000 7f68b7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f68b8072fe0 con 0x7f68b80cd180 2024-04-28T12:43:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.292+0000 7f68b7400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f68b80ce480 0x7f68b806dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54136/0 (socket says 172.21.15.131:54136) 2024-04-28T12:43:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.292+0000 7f68b7400640 1 -- 172.21.15.131:0/1571063159 learned_addr learned my addr 172.21.15.131:0/1571063159 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.292+0000 7f68b6a00640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68b80cd180 0x7f68b8072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.292+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3594508847 0 0) 0x7f68b80cd560 con 0x7f68b80ce480 2024-04-28T12:43:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f689c003520 con 0x7f68b80ce480 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3354821210 0 0) 0x7f689c003520 con 0x7f68b80ce480 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68b80cd180 msgr2=0x7f68b8072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68b80cd180 0x7f68b8072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 >> v1:172.21.15.157:6789/0 conn(0x7f68b815dc50 legacy=0x7f68b80cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b80732c0 con 0x7f68b80ce480 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f68b00019f0 con 0x7f68b80ce480 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 >> v1:172.21.15.131:6789/0 conn(0x7f68b80ce480 legacy=0x7f68b806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f689c0039e0 0x7f689c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f689c004800 0x7f689c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68b80cd180 0x7f689c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8072fe0 con 0x7f689c004800 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f689c00ba70 con 0x7f689c0039e0 2024-04-28T12:43:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f689c00bc70 con 0x7f68b80cd180 2024-04-28T12:43:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.293+0000 7f68b6000640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f689c004800 0x7f689c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.294+0000 7f68b6a00640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f689c0039e0 0x7f689c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.294+0000 7f68b7400640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68b80cd180 0x7f689c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.294+0000 7f68b6000640 1 -- 172.21.15.131:0/1571063159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68b80cd180 msgr2=0x7f689c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.294+0000 7f68b6000640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68b80cd180 0x7f689c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.294+0000 7f68b6000640 1 -- 172.21.15.131:0/1571063159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f689c0039e0 msgr2=0x7f689c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.294+0000 7f68b6000640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f689c0039e0 0x7f689c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.294+0000 7f68b6000640 1 -- 172.21.15.131:0/1571063159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c003520 con 0x7f689c004800 2024-04-28T12:43:10.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.294+0000 7f68b7400640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68b80cd180 0x7f689c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.295+0000 7f68b6000640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f689c004800 0x7f689c004c80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f68a4001720 tx=0x7f689c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1151494219fc467e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.295+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4001370 con 0x7f689c004800 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.295+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68a40052d0 con 0x7f689c004800 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.295+0000 7f68b5600640 1 -- 172.21.15.131:0/1571063159 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a402ab90 con 0x7f689c004800 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.296+0000 7f68b7e00640 1 -- 172.21.15.131:0/1571063159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f689c004800 msgr2=0x7f689c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.296+0000 7f68b7e00640 1 --2- 172.21.15.131:0/1571063159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f689c004800 0x7f689c004c80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f68a4001720 tx=0x7f689c005830 comp rx=0 tx=0).stop 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.296+0000 7f68b7400640 1 -- 172.21.15.131:0/1571063159 reap_dead start 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.296+0000 7f68b7e00640 1 -- 172.21.15.131:0/1571063159 shutdown_connections 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.296+0000 7f68b7e00640 1 -- 172.21.15.131:0/1571063159 >> 172.21.15.131:0/1571063159 conn(0x7f68b8068090 msgr2=0x7f68b8068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.296+0000 7f68b7e00640 1 -- 172.21.15.131:0/1571063159 shutdown_connections 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.296+0000 7f68b7e00640 1 -- 172.21.15.131:0/1571063159 wait complete. 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.298+0000 7f68b7e00640 1 Processor -- start 2024-04-28T12:43:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.298+0000 7f68b7e00640 1 -- start start 2024-04-28T12:43:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f689c004800 0x7f68b81f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68b80af9e0 0x7f68b81422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b813ffb0 0x7f68b81e8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f68b815e0d0 con 0x7f68b813ffb0 2024-04-28T12:43:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8073da0 con 0x7f68b80af9e0 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f68b8072620 con 0x7f689c004800 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b813ffb0 0x7f68b81e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b813ffb0 0x7f68b81e8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53926/0 (socket says 172.21.15.131:53926) 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7400640 1 -- 172.21.15.131:0/1384039937 learned_addr learned my addr 172.21.15.131:0/1384039937 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b6a00640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f689c004800 0x7f68b81f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7400640 1 -- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f689c004800 msgr2=0x7f68b81f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7400640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f689c004800 0x7f68b81f32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b6000640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68b80af9e0 0x7f68b81422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7400640 1 -- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68b80af9e0 msgr2=0x7f68b81422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7400640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68b80af9e0 0x7f68b81422f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.299+0000 7f68b7400640 1 -- 172.21.15.131:0/1384039937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b80aa7b0 con 0x7f68b813ffb0 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.300+0000 7f68b7400640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b813ffb0 0x7f68b81e8030 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f68b80720b0 tx=0x7f68b000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.300+0000 7f68aea00640 1 -- 172.21.15.131:0/1384039937 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b000de90 con 0x7f68b813ffb0 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.300+0000 7f68aea00640 1 -- 172.21.15.131:0/1384039937 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b000bb80 con 0x7f68b813ffb0 2024-04-28T12:43:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.300+0000 7f68b7e00640 1 -- 172.21.15.131:0/1384039937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b81f3960 con 0x7f68b813ffb0 2024-04-28T12:43:10.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.300+0000 7f68aea00640 1 -- 172.21.15.131:0/1384039937 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0007110 con 0x7f68b813ffb0 2024-04-28T12:43:10.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.300+0000 7f68b7e00640 1 -- 172.21.15.131:0/1384039937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b81f3f20 con 0x7f68b813ffb0 2024-04-28T12:43:10.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.301+0000 7f68b7e00640 1 -- 172.21.15.131:0/1384039937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b8069350 con 0x7f68b813ffb0 2024-04-28T12:43:10.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.302+0000 7f68aea00640 1 -- 172.21.15.131:0/1384039937 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68b001b060 con 0x7f68b813ffb0 2024-04-28T12:43:10.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.303+0000 7f68aea00640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f689806ca90 0x7f689806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.303+0000 7f68aea00640 1 -- 172.21.15.131:0/1384039937 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f68b008bd90 con 0x7f68b813ffb0 2024-04-28T12:43:10.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.303+0000 7f68b6a00640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f689806ca90 0x7f689806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.305+0000 7f68b6a00640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f689806ca90 0x7f689806ef50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f68b815f0d0 tx=0x7f68a8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:10.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.307+0000 7f68aea00640 1 -- 172.21.15.131:0/1384039937 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f68b00599b0 con 0x7f68b813ffb0 2024-04-28T12:43:10.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.547+0000 7f68b7e00640 1 -- 172.21.15.131:0/1384039937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68b80d0670 con 0x7f68b813ffb0 2024-04-28T12:43:10.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.548+0000 7f68aea00640 1 -- 172.21.15.131:0/1384039937 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5189 (secure 0 0 0) 0x7f68b005dcb0 con 0x7f68b813ffb0 2024-04-28T12:43:10.544 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:10.544 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/361318591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":361318591},{"type":"v1","addr":"172.21.15.157:6836","nonce":361318591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/102335005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":102335005},{"type":"v1","addr":"172.21.15.131:6835","nonce":102335005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1609059776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1609059776},{"type":"v1","addr":"172.21.15.131:6833","nonce":1609059776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:39:10.533686+0000","modified":"2024-04-28T12:39:13.595067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34610":{"gid":34610,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/837933884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":837933884},{"type":"v1","addr":"172.21.15.157:6837","nonce":837933884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34610,"qdb_cluster":[34610]},"id":2}]} 2024-04-28T12:43:10.545 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2024-04-28T12:43:10.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.551+0000 7f68b7e00640 1 -- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f689806ca90 msgr2=0x7f689806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.551+0000 7f68b7e00640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f689806ca90 0x7f689806ef50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f68b815f0d0 tx=0x7f68a8008040 comp rx=0 tx=0).stop 2024-04-28T12:43:10.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.551+0000 7f68b7e00640 1 -- 172.21.15.131:0/1384039937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b813ffb0 msgr2=0x7f68b81e8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.551+0000 7f68b7e00640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b813ffb0 0x7f68b81e8030 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f68b80720b0 tx=0x7f68b000b4a0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.551+0000 7f68b7e00640 1 -- 172.21.15.131:0/1384039937 shutdown_connections 2024-04-28T12:43:10.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.551+0000 7f68b7e00640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f689806ca90 0x7f689806ef50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.551+0000 7f68b7e00640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b813ffb0 0x7f68b81e8030 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.551+0000 7f68b7e00640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68b80af9e0 0x7f68b81422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.551+0000 7f68b7e00640 1 --2- 172.21.15.131:0/1384039937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f689c004800 0x7f68b81f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.551+0000 7f68b7e00640 1 -- 172.21.15.131:0/1384039937 >> 172.21.15.131:0/1384039937 conn(0x7f68b8068090 msgr2=0x7f68b80b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:10.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.551+0000 7f68b7e00640 1 -- 172.21.15.131:0/1384039937 shutdown_connections 2024-04-28T12:43:10.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.551+0000 7f68b7e00640 1 -- 172.21.15.131:0/1384039937 wait complete. 2024-04-28T12:43:10.566 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:43:10.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.751+0000 7fb71b200640 1 Processor -- start 2024-04-28T12:43:10.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.751+0000 7fb71b200640 1 -- start start 2024-04-28T12:43:10.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.753+0000 7fb71b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c0d5750 0x7fb71c0d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.753+0000 7fb71b200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71c0d0480 con 0x7fb71c0d1620 2024-04-28T12:43:10.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.753+0000 7fb71b200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71c0d0720 con 0x7fb71c0b5b80 2024-04-28T12:43:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.753+0000 7fb71b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c0d5be0 con 0x7fb71c0d5750 2024-04-28T12:43:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.753+0000 7fb719e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb71c0d1620 0x7fb71c0d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54150/0 (socket says 172.21.15.131:54150) 2024-04-28T12:43:10.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.753+0000 7fb719e00640 1 -- 172.21.15.131:0/707813244 learned_addr learned my addr 172.21.15.131:0/707813244 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:10.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 208928224 0 0) 0x7fb71c0d0480 con 0x7fb71c0d1620 2024-04-28T12:43:10.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6fc003520 con 0x7fb71c0d1620 2024-04-28T12:43:10.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb71a800640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c0d5750 0x7fb71c0d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 615417180 0 0) 0x7fb6fc003520 con 0x7fb71c0d1620 2024-04-28T12:43:10.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c0d5750 msgr2=0x7fb71c0d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:10.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c0d5750 0x7fb71c0d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 >> v1:172.21.15.157:6789/0 conn(0x7fb71c0b5b80 legacy=0x7fb71c0d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:10.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c0d0a00 con 0x7fb71c0d1620 2024-04-28T12:43:10.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fb70c001670 con 0x7fb71c0d1620 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 >> v1:172.21.15.131:6789/0 conn(0x7fb71c0d1620 legacy=0x7fb71c0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc0039e0 0x7fb6fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6fc004800 0x7fb6fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6fc0052e0 0x7fb6fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c0d5be0 con 0x7fb6fc0052e0 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc00c1e0 con 0x7fb6fc0039e0 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb6fc00c3e0 con 0x7fb6fc004800 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb71a800640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c0d5750 0x7fb71c0d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.754+0000 7fb71a800640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6fc0052e0 0x7fb6fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.755+0000 7fb719e00640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc0039e0 0x7fb6fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.755+0000 7fb719400640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6fc004800 0x7fb6fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.755+0000 7fb71a800640 1 -- 172.21.15.131:0/707813244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6fc004800 msgr2=0x7fb6fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:10.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.755+0000 7fb71a800640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6fc004800 0x7fb6fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.755+0000 7fb71a800640 1 -- 172.21.15.131:0/707813244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc0039e0 msgr2=0x7fb6fc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.755+0000 7fb71a800640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6fc0039e0 0x7fb6fc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.755+0000 7fb71a800640 1 -- 172.21.15.131:0/707813244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc003520 con 0x7fb6fc0052e0 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.756+0000 7fb71a800640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6fc0052e0 0x7fb6fc00b9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb71400a860 tx=0x7fb6fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f36b91366c1434 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.756+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714037070 con 0x7fb6fc0052e0 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.756+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71402aa90 con 0x7fb6fc0052e0 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.756+0000 7fb718a00640 1 -- 172.21.15.131:0/707813244 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71400a4b0 con 0x7fb6fc0052e0 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.757+0000 7fb71b200640 1 -- 172.21.15.131:0/707813244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6fc0052e0 msgr2=0x7fb6fc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.757+0000 7fb71b200640 1 --2- 172.21.15.131:0/707813244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6fc0052e0 0x7fb6fc00b9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb71400a860 tx=0x7fb6fc005830 comp rx=0 tx=0).stop 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.757+0000 7fb71a800640 1 -- 172.21.15.131:0/707813244 reap_dead start 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.757+0000 7fb71b200640 1 -- 172.21.15.131:0/707813244 shutdown_connections 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.757+0000 7fb71b200640 1 -- 172.21.15.131:0/707813244 >> 172.21.15.131:0/707813244 conn(0x7fb71c0d4c30 msgr2=0x7fb71c0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.757+0000 7fb71b200640 1 -- 172.21.15.131:0/707813244 shutdown_connections 2024-04-28T12:43:10.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.757+0000 7fb71b200640 1 -- 172.21.15.131:0/707813244 wait complete. 2024-04-28T12:43:10.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.758+0000 7fb71b200640 1 Processor -- start 2024-04-28T12:43:10.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.758+0000 7fb71b200640 1 -- start start 2024-04-28T12:43:10.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.759+0000 7fb71b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71c0ca940 0x7fb71c15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.759+0000 7fb71b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c159950 0x7fb71c0b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.759+0000 7fb71b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c0b8250 0x7fb71c0b9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.759+0000 7fb71b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c0d2f10 con 0x7fb71c0ca940 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.759+0000 7fb71b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c0d0ce0 con 0x7fb71c159950 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.759+0000 7fb71b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c0d3090 con 0x7fb71c0b8250 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb719e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71c0ca940 0x7fb71c15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb719e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71c0ca940 0x7fb71c15ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53958/0 (socket says 172.21.15.131:53958) 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb719e00640 1 -- 172.21.15.131:0/2141673460 learned_addr learned my addr 172.21.15.131:0/2141673460 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb719e00640 1 -- 172.21.15.131:0/2141673460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c0b8250 msgr2=0x7fb71c0b9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb719e00640 1 --2- 172.21.15.131:0/2141673460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c0b8250 0x7fb71c0b9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb719e00640 1 -- 172.21.15.131:0/2141673460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c159950 msgr2=0x7fb71c0b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb719e00640 1 --2- 172.21.15.131:0/2141673460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c159950 0x7fb71c0b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb719e00640 1 -- 172.21.15.131:0/2141673460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c0cad20 con 0x7fb71c0ca940 2024-04-28T12:43:10.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb719e00640 1 --2- 172.21.15.131:0/2141673460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71c0ca940 0x7fb71c15ae80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00e8c0 tx=0x7fb70c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:10.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb712000640 1 -- 172.21.15.131:0/2141673460 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c00c7f0 con 0x7fb71c0ca940 2024-04-28T12:43:10.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb712000640 1 -- 172.21.15.131:0/2141673460 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70c018070 con 0x7fb71c0ca940 2024-04-28T12:43:10.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb712000640 1 -- 172.21.15.131:0/2141673460 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c00cb90 con 0x7fb71c0ca940 2024-04-28T12:43:10.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.760+0000 7fb71b200640 1 -- 172.21.15.131:0/2141673460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c159de0 con 0x7fb71c0ca940 2024-04-28T12:43:10.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.761+0000 7fb71b200640 1 -- 172.21.15.131:0/2141673460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c15a3a0 con 0x7fb71c0ca940 2024-04-28T12:43:10.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.762+0000 7fb71b200640 1 -- 172.21.15.131:0/2141673460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c069350 con 0x7fb71c0ca940 2024-04-28T12:43:10.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.763+0000 7fb712000640 1 -- 172.21.15.131:0/2141673460 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb70c01d460 con 0x7fb71c0ca940 2024-04-28T12:43:10.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.763+0000 7fb712000640 1 --2- 172.21.15.131:0/2141673460 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb6f806c8f0 0x7fb6f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:10.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.763+0000 7fb712000640 1 -- 172.21.15.131:0/2141673460 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fb70c08bd90 con 0x7fb71c0ca940 2024-04-28T12:43:10.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.764+0000 7fb719400640 1 --2- 172.21.15.131:0/2141673460 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb6f806c8f0 0x7fb6f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:10.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.765+0000 7fb719400640 1 --2- 172.21.15.131:0/2141673460 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb6f806c8f0 0x7fb6f806edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb704007fd0 tx=0x7fb704009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:10.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:10.768+0000 7fb712000640 1 -- 172.21.15.131:0/2141673460 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb70c059ee0 con 0x7fb71c0ca940 2024-04-28T12:43:11.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.010+0000 7fb71b200640 1 -- 172.21.15.131:0/2141673460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb71c0baa10 con 0x7fb71c0ca940 2024-04-28T12:43:11.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.012+0000 7fb712000640 1 -- 172.21.15.131:0/2141673460 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5189 (secure 0 0 0) 0x7fb70c0597e0 con 0x7fb71c0ca940 2024-04-28T12:43:11.008 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:11.008 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/361318591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":361318591},{"type":"v1","addr":"172.21.15.157:6836","nonce":361318591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/102335005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":102335005},{"type":"v1","addr":"172.21.15.131:6835","nonce":102335005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1609059776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1609059776},{"type":"v1","addr":"172.21.15.131:6833","nonce":1609059776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:39:10.533686+0000","modified":"2024-04-28T12:39:13.595067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34610":{"gid":34610,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6837/837933884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":837933884},{"type":"v1","addr":"172.21.15.157:6837","nonce":837933884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34610,"qdb_cluster":[34610]},"id":2}]} 2024-04-28T12:43:11.008 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2024-04-28T12:43:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.014+0000 7fb71b200640 1 -- 172.21.15.131:0/2141673460 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb6f806c8f0 msgr2=0x7fb6f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.014+0000 7fb71b200640 1 --2- 172.21.15.131:0/2141673460 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb6f806c8f0 0x7fb6f806edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb704007fd0 tx=0x7fb704009040 comp rx=0 tx=0).stop 2024-04-28T12:43:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.014+0000 7fb71b200640 1 -- 172.21.15.131:0/2141673460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71c0ca940 msgr2=0x7fb71c15ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.014+0000 7fb71b200640 1 --2- 172.21.15.131:0/2141673460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71c0ca940 0x7fb71c15ae80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00e8c0 tx=0x7fb70c00ed80 comp rx=0 tx=0).stop 2024-04-28T12:43:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.015+0000 7fb71b200640 1 -- 172.21.15.131:0/2141673460 shutdown_connections 2024-04-28T12:43:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.015+0000 7fb71b200640 1 --2- 172.21.15.131:0/2141673460 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fb6f806c8f0 0x7fb6f806edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.015+0000 7fb71b200640 1 --2- 172.21.15.131:0/2141673460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c0b8250 0x7fb71c0b9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.015+0000 7fb71b200640 1 --2- 172.21.15.131:0/2141673460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c159950 0x7fb71c0b9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.015+0000 7fb71b200640 1 --2- 172.21.15.131:0/2141673460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71c0ca940 0x7fb71c15ae80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.015+0000 7fb71b200640 1 -- 172.21.15.131:0/2141673460 >> 172.21.15.131:0/2141673460 conn(0x7fb71c0d4c30 msgr2=0x7fb71c074ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:11.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.015+0000 7fb71b200640 1 -- 172.21.15.131:0/2141673460 shutdown_connections 2024-04-28T12:43:11.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.015+0000 7fb71b200640 1 -- 172.21.15.131:0/2141673460 wait complete. 2024-04-28T12:43:11.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:43:11.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.232+0000 7f18c9800640 1 Processor -- start 2024-04-28T12:43:11.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.232+0000 7f18c9800640 1 -- start start 2024-04-28T12:43:11.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.234+0000 7f18c9800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18c40cd180 0x7f18c4072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:11.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.234+0000 7f18c9800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f18c415dc50 0x7f18c40cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:11.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.234+0000 7f18c9800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c40cd560 con 0x7f18c40ce2b0 2024-04-28T12:43:11.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.234+0000 7f18c9800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f18c4072d00 con 0x7f18c415dc50 2024-04-28T12:43:11.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.234+0000 7f18c9800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f18c4072fe0 con 0x7f18c40cd180 2024-04-28T12:43:11.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.234+0000 7f18c3400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f18c415dc50 0x7f18c40cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:11.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.234+0000 7f18c3400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f18c415dc50 0x7f18c40cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33706/0 (socket says 172.21.15.131:33706) 2024-04-28T12:43:11.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.234+0000 7f18c3400640 1 -- 172.21.15.131:0/834644981 learned_addr learned my addr 172.21.15.131:0/834644981 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:11.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.234+0000 7f18c3e00640 1 --2- 172.21.15.131:0/834644981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18c40cd180 0x7f18c4072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:11.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.235+0000 7f18c3400640 1 -- 172.21.15.131:0/834644981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18c40cd180 msgr2=0x7f18c4072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:11.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.235+0000 7f18c3400640 1 --2- 172.21.15.131:0/834644981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18c40cd180 0x7f18c4072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.235+0000 7f18c3400640 1 -- 172.21.15.131:0/834644981 >> v1:172.21.15.157:6789/0 conn(0x7f18c40ce2b0 legacy=0x7f18c41607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:11.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.235+0000 7f18c3400640 1 -- 172.21.15.131:0/834644981 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c40732c0 con 0x7f18c415dc50 2024-04-28T12:43:11.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.236+0000 7f18c3400640 1 --2- 172.21.15.131:0/834644981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c415dc50 0x7f18c40cdd70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009870 tx=0x7f18b002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28d5d3ef397fc25 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:11.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.236+0000 7f18c2a00640 1 -- 172.21.15.131:0/834644981 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0037070 con 0x7f18c415dc50 2024-04-28T12:43:11.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.236+0000 7f18c2a00640 1 -- 172.21.15.131:0/834644981 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b0032440 con 0x7f18c415dc50 2024-04-28T12:43:11.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.236+0000 7f18c2a00640 1 -- 172.21.15.131:0/834644981 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0032810 con 0x7f18c415dc50 2024-04-28T12:43:11.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.237+0000 7f18c9800640 1 -- 172.21.15.131:0/834644981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c415dc50 msgr2=0x7f18c40cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.237+0000 7f18c9800640 1 --2- 172.21.15.131:0/834644981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c415dc50 0x7f18c40cdd70 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009870 tx=0x7f18b002a200 comp rx=0 tx=0).stop 2024-04-28T12:43:11.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.237+0000 7f18c9800640 1 -- 172.21.15.131:0/834644981 shutdown_connections 2024-04-28T12:43:11.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.237+0000 7f18c9800640 1 --2- 172.21.15.131:0/834644981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c415dc50 0x7f18c40cdd70 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.237+0000 7f18c9800640 1 --2- 172.21.15.131:0/834644981 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18c40cd180 0x7f18c4072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.237+0000 7f18c9800640 1 -- 172.21.15.131:0/834644981 >> 172.21.15.131:0/834644981 conn(0x7f18c4068090 msgr2=0x7f18c4068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:11.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.237+0000 7f18c9800640 1 -- 172.21.15.131:0/834644981 shutdown_connections 2024-04-28T12:43:11.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.237+0000 7f18c9800640 1 -- 172.21.15.131:0/834644981 wait complete. 2024-04-28T12:43:11.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.239+0000 7f18c9800640 1 Processor -- start 2024-04-28T12:43:11.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.239+0000 7f18c9800640 1 -- start start 2024-04-28T12:43:11.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.240+0000 7f18c9800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18c40cd180 0x7f18c41f9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:11.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.240+0000 7f18c9800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c40ce2b0 0x7f18c415c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:11.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.240+0000 7f18c9800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18c415dc50 0x7f18c40c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:11.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.240+0000 7f18c9800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f18c406ead0 con 0x7f18c40ce2b0 2024-04-28T12:43:11.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.240+0000 7f18c9800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4073da0 con 0x7f18c415dc50 2024-04-28T12:43:11.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.240+0000 7f18c9800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f18c415e0d0 con 0x7f18c40cd180 2024-04-28T12:43:11.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.240+0000 7f18c3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c40ce2b0 0x7f18c415c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:11.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.240+0000 7f18c3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c40ce2b0 0x7f18c415c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33708/0 (socket says 172.21.15.131:33708) 2024-04-28T12:43:11.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.240+0000 7f18c3400640 1 -- 172.21.15.131:0/2118545208 learned_addr learned my addr 172.21.15.131:0/2118545208 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:11.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.241+0000 7f18c3400640 1 -- 172.21.15.131:0/2118545208 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18c40cd180 msgr2=0x7f18c41f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.241+0000 7f18c3400640 1 --2- 172.21.15.131:0/2118545208 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18c40cd180 0x7f18c41f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.241+0000 7f18c3400640 1 -- 172.21.15.131:0/2118545208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18c415dc50 msgr2=0x7f18c40c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.241+0000 7f18c3400640 1 --2- 172.21.15.131:0/2118545208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18c415dc50 0x7f18c40c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.241+0000 7f18c3400640 1 -- 172.21.15.131:0/2118545208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c41f6020 con 0x7f18c40ce2b0 2024-04-28T12:43:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.241+0000 7f18c3400640 1 --2- 172.21.15.131:0/2118545208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c40ce2b0 0x7f18c415c620 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f18b0000c00 tx=0x7f18b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.241+0000 7f18bbe00640 1 -- 172.21.15.131:0/2118545208 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0041020 con 0x7f18c40ce2b0 2024-04-28T12:43:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.241+0000 7f18bbe00640 1 -- 172.21.15.131:0/2118545208 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b00306c0 con 0x7f18c40ce2b0 2024-04-28T12:43:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.241+0000 7f18c9800640 1 -- 172.21.15.131:0/2118545208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c41f5d40 con 0x7f18c40ce2b0 2024-04-28T12:43:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.241+0000 7f18bbe00640 1 -- 172.21.15.131:0/2118545208 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0037040 con 0x7f18c40ce2b0 2024-04-28T12:43:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.241+0000 7f18c9800640 1 -- 172.21.15.131:0/2118545208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c41f8a40 con 0x7f18c40ce2b0 2024-04-28T12:43:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.242+0000 7f18c9800640 1 -- 172.21.15.131:0/2118545208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c40ce0d0 con 0x7f18c40ce2b0 2024-04-28T12:43:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.243+0000 7f18bbe00640 1 -- 172.21.15.131:0/2118545208 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f18b0044020 con 0x7f18c40ce2b0 2024-04-28T12:43:11.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.245+0000 7f18bbe00640 1 --2- 172.21.15.131:0/2118545208 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f18a406c8f0 0x7f18a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:11.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.245+0000 7f18bbe00640 1 -- 172.21.15.131:0/2118545208 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f18b00ac520 con 0x7f18c40ce2b0 2024-04-28T12:43:11.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.245+0000 7f18c3e00640 1 --2- 172.21.15.131:0/2118545208 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f18a406c8f0 0x7f18a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:11.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.246+0000 7f18c3e00640 1 --2- 172.21.15.131:0/2118545208 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f18a406c8f0 0x7f18a406edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f18b40098f0 tx=0x7f18b4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:11.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.249+0000 7f18bbe00640 1 -- 172.21.15.131:0/2118545208 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f18c40ce0d0 con 0x7f18c40ce2b0 2024-04-28T12:43:11.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.449+0000 7f18c9800640 1 -- 172.21.15.131:0/2118545208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18c4069350 con 0x7f18c40ce2b0 2024-04-28T12:43:11.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.452+0000 7f18bbe00640 1 -- 172.21.15.131:0/2118545208 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16958 (secure 0 0 0) 0x7f18b004b030 con 0x7f18c40ce2b0 2024-04-28T12:43:11.449 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:11.449 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:40:08.292981+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:39:08.310817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-28T12:39:09.505586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:43:11.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.455+0000 7f18c9800640 1 -- 172.21.15.131:0/2118545208 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f18a406c8f0 msgr2=0x7f18a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.455+0000 7f18c9800640 1 --2- 172.21.15.131:0/2118545208 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f18a406c8f0 0x7f18a406edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f18b40098f0 tx=0x7f18b4008040 comp rx=0 tx=0).stop 2024-04-28T12:43:11.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.455+0000 7f18c9800640 1 -- 172.21.15.131:0/2118545208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c40ce2b0 msgr2=0x7f18c415c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.455+0000 7f18c9800640 1 --2- 172.21.15.131:0/2118545208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c40ce2b0 0x7f18c415c620 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f18b0000c00 tx=0x7f18b00046c0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.455+0000 7f18c9800640 1 -- 172.21.15.131:0/2118545208 shutdown_connections 2024-04-28T12:43:11.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.455+0000 7f18c9800640 1 --2- 172.21.15.131:0/2118545208 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f18a406c8f0 0x7f18a406edb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.455+0000 7f18c9800640 1 --2- 172.21.15.131:0/2118545208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18c415dc50 0x7f18c40c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.455+0000 7f18c9800640 1 --2- 172.21.15.131:0/2118545208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c40ce2b0 0x7f18c415c620 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.455+0000 7f18c9800640 1 --2- 172.21.15.131:0/2118545208 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f18c40cd180 0x7f18c41f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.455+0000 7f18c9800640 1 -- 172.21.15.131:0/2118545208 >> 172.21.15.131:0/2118545208 conn(0x7f18c4068090 msgr2=0x7f18c4160760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:11.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.455+0000 7f18c9800640 1 -- 172.21.15.131:0/2118545208 shutdown_connections 2024-04-28T12:43:11.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.455+0000 7f18c9800640 1 -- 172.21.15.131:0/2118545208 wait complete. 2024-04-28T12:43:11.471 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-28T12:43:11.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.677+0000 7f94c7a00640 1 Processor -- start 2024-04-28T12:43:11.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.677+0000 7f94c7a00640 1 -- start start 2024-04-28T12:43:11.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.679+0000 7f94c7a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c0072b70 0x7f94c00d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:11.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.679+0000 7f94c7a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f94c00d1060 0x7f94c006dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:11.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.679+0000 7f94c7a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c0146ba0 con 0x7f94c01467c0 2024-04-28T12:43:11.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.679+0000 7f94c7a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f94c0073dd0 con 0x7f94c00d1060 2024-04-28T12:43:11.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.679+0000 7f94c7a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f94c00731c0 con 0x7f94c0072b70 2024-04-28T12:43:11.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.679+0000 7f94c5a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f94c00d1060 0x7f94c006dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:11.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.679+0000 7f94c5a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f94c00d1060 0x7f94c006dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33722/0 (socket says 172.21.15.131:33722) 2024-04-28T12:43:11.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.679+0000 7f94c5a00640 1 -- 172.21.15.131:0/1512456834 learned_addr learned my addr 172.21.15.131:0/1512456834 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:11.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.680+0000 7f94bfe00640 1 --2- 172.21.15.131:0/1512456834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c0072b70 0x7f94c00d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:11.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.680+0000 7f94c5a00640 1 -- 172.21.15.131:0/1512456834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c0072b70 msgr2=0x7f94c00d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:11.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.680+0000 7f94c5a00640 1 --2- 172.21.15.131:0/1512456834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c0072b70 0x7f94c00d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.680+0000 7f94c5a00640 1 -- 172.21.15.131:0/1512456834 >> v1:172.21.15.157:6789/0 conn(0x7f94c01467c0 legacy=0x7f94c00b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:11.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.680+0000 7f94c5a00640 1 -- 172.21.15.131:0/1512456834 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c00734a0 con 0x7f94c00d1060 2024-04-28T12:43:11.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.681+0000 7f94c5a00640 1 --2- 172.21.15.131:0/1512456834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94c00d1060 0x7f94c006dff0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f94c0073f50 tx=0x7f94b802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d851a6b8537a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:11.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.681+0000 7f94bf400640 1 -- 172.21.15.131:0/1512456834 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8036070 con 0x7f94c00d1060 2024-04-28T12:43:11.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.682+0000 7f94bf400640 1 -- 172.21.15.131:0/1512456834 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b8031600 con 0x7f94c00d1060 2024-04-28T12:43:11.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.682+0000 7f94bf400640 1 -- 172.21.15.131:0/1512456834 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b80319d0 con 0x7f94c00d1060 2024-04-28T12:43:11.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.682+0000 7f94c7a00640 1 -- 172.21.15.131:0/1512456834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94c00d1060 msgr2=0x7f94c006dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.682+0000 7f94c7a00640 1 --2- 172.21.15.131:0/1512456834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94c00d1060 0x7f94c006dff0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f94c0073f50 tx=0x7f94b802a710 comp rx=0 tx=0).stop 2024-04-28T12:43:11.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.682+0000 7f94c7a00640 1 -- 172.21.15.131:0/1512456834 shutdown_connections 2024-04-28T12:43:11.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.682+0000 7f94c7a00640 1 --2- 172.21.15.131:0/1512456834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94c00d1060 0x7f94c006dff0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.682+0000 7f94c7a00640 1 --2- 172.21.15.131:0/1512456834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c0072b70 0x7f94c00d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.682+0000 7f94c7a00640 1 -- 172.21.15.131:0/1512456834 >> 172.21.15.131:0/1512456834 conn(0x7f94c00b2900 msgr2=0x7f94c00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:11.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.682+0000 7f94c7a00640 1 -- 172.21.15.131:0/1512456834 shutdown_connections 2024-04-28T12:43:11.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.682+0000 7f94c7a00640 1 -- 172.21.15.131:0/1512456834 wait complete. 2024-04-28T12:43:11.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.684+0000 7f94c7a00640 1 Processor -- start 2024-04-28T12:43:11.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.684+0000 7f94c7a00640 1 -- start start 2024-04-28T12:43:11.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.685+0000 7f94c7a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c0072b70 0x7f94c015fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:11.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.685+0000 7f94c7a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94c00d1060 0x7f94c01e6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:11.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.685+0000 7f94c7a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94c01467c0 0x7f94c01e6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:11.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.685+0000 7f94c7a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f94c006e530 con 0x7f94c00d1060 2024-04-28T12:43:11.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.685+0000 7f94c7a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0073a60 con 0x7f94c01467c0 2024-04-28T12:43:11.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.685+0000 7f94c7a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f94c006e6b0 con 0x7f94c0072b70 2024-04-28T12:43:11.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.685+0000 7f94bfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94c00d1060 0x7f94c01e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:11.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.685+0000 7f94bfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94c00d1060 0x7f94c01e6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33728/0 (socket says 172.21.15.131:33728) 2024-04-28T12:43:11.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.685+0000 7f94bfe00640 1 -- 172.21.15.131:0/153357460 learned_addr learned my addr 172.21.15.131:0/153357460 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:11.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.686+0000 7f94bfe00640 1 -- 172.21.15.131:0/153357460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c0072b70 msgr2=0x7f94c015fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.686+0000 7f94bfe00640 1 --2- 172.21.15.131:0/153357460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c0072b70 0x7f94c015fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.686+0000 7f94bfe00640 1 -- 172.21.15.131:0/153357460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94c01467c0 msgr2=0x7f94c01e6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.686+0000 7f94bfe00640 1 --2- 172.21.15.131:0/153357460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94c01467c0 0x7f94c01e6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.686+0000 7f94bfe00640 1 -- 172.21.15.131:0/153357460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c0132c80 con 0x7f94c00d1060 2024-04-28T12:43:11.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.686+0000 7f94bfe00640 1 --2- 172.21.15.131:0/153357460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94c00d1060 0x7f94c01e6980 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00d800 tx=0x7f94ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:11.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.686+0000 7f94bcc00640 1 -- 172.21.15.131:0/153357460 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac00b6c0 con 0x7f94c00d1060 2024-04-28T12:43:11.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.686+0000 7f94bcc00640 1 -- 172.21.15.131:0/153357460 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ac019070 con 0x7f94c00d1060 2024-04-28T12:43:11.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.686+0000 7f94c7a00640 1 -- 172.21.15.131:0/153357460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c0132f60 con 0x7f94c00d1060 2024-04-28T12:43:11.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.686+0000 7f94bcc00640 1 -- 172.21.15.131:0/153357460 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac00ba80 con 0x7f94c00d1060 2024-04-28T12:43:11.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.687+0000 7f94c7a00640 1 -- 172.21.15.131:0/153357460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c0133520 con 0x7f94c00d1060 2024-04-28T12:43:11.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.688+0000 7f94c7a00640 1 -- 172.21.15.131:0/153357460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c0069350 con 0x7f94c00d1060 2024-04-28T12:43:11.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.689+0000 7f94bcc00640 1 -- 172.21.15.131:0/153357460 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94ac00bc20 con 0x7f94c00d1060 2024-04-28T12:43:11.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.689+0000 7f94bcc00640 1 --2- 172.21.15.131:0/153357460 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f94a006c8f0 0x7f94a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:11.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.690+0000 7f94bcc00640 1 -- 172.21.15.131:0/153357460 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f94ac08c120 con 0x7f94c00d1060 2024-04-28T12:43:11.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.690+0000 7f94c5000640 1 --2- 172.21.15.131:0/153357460 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f94a006c8f0 0x7f94a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:11.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.691+0000 7f94c5000640 1 --2- 172.21.15.131:0/153357460 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f94a006c8f0 0x7f94a006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f94b0000f60 tx=0x7f94b00079d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:11.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.694+0000 7f94bcc00640 1 -- 172.21.15.131:0/153357460 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94ac011340 con 0x7f94c00d1060 2024-04-28T12:43:11.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.933+0000 7f94c7a00640 1 -- 172.21.15.131:0/153357460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f94c00b5480 con 0x7f94c00d1060 2024-04-28T12:43:11.930 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-04-28T12:43:11.934+0000 7f10b6000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-28T12:43:11.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.938+0000 7f94bcc00640 1 -- 172.21.15.131:0/153357460 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f94ac05a430 con 0x7f94c00d1060 2024-04-28T12:43:11.934 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-28T12:43:11.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.941+0000 7f94c7a00640 1 -- 172.21.15.131:0/153357460 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f94a006c8f0 msgr2=0x7f94a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.941+0000 7f94c7a00640 1 --2- 172.21.15.131:0/153357460 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f94a006c8f0 0x7f94a006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f94b0000f60 tx=0x7f94b00079d0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.941+0000 7f94c7a00640 1 -- 172.21.15.131:0/153357460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94c00d1060 msgr2=0x7f94c01e6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.941+0000 7f94c7a00640 1 --2- 172.21.15.131:0/153357460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94c00d1060 0x7f94c01e6980 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00d800 tx=0x7f94ac00dcc0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.941+0000 7f94c7a00640 1 -- 172.21.15.131:0/153357460 shutdown_connections 2024-04-28T12:43:11.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.941+0000 7f94c7a00640 1 --2- 172.21.15.131:0/153357460 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f94a006c8f0 0x7f94a006edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.941+0000 7f94c7a00640 1 --2- 172.21.15.131:0/153357460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94c01467c0 0x7f94c01e6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.941+0000 7f94c7a00640 1 --2- 172.21.15.131:0/153357460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94c00d1060 0x7f94c01e6980 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.941+0000 7f94c7a00640 1 --2- 172.21.15.131:0/153357460 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c0072b70 0x7f94c015fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.941+0000 7f94c7a00640 1 -- 172.21.15.131:0/153357460 >> 172.21.15.131:0/153357460 conn(0x7f94c00b2900 msgr2=0x7f94c00b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:11.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.941+0000 7f94c7a00640 1 -- 172.21.15.131:0/153357460 shutdown_connections 2024-04-28T12:43:11.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:11.941+0000 7f94c7a00640 1 -- 172.21.15.131:0/153357460 wait complete. 2024-04-28T12:43:11.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-28T12:43:12.054 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-04-28T12:43:12.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.161+0000 7f7181e00640 1 Processor -- start 2024-04-28T12:43:12.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.162+0000 7f7181e00640 1 -- start start 2024-04-28T12:43:12.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.163+0000 7f7181e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717c0d31d0 0x7f717c0d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:12.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.163+0000 7f7181e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f717c0b5160 con 0x7f717c0d3780 2024-04-28T12:43:12.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.163+0000 7f7181e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f717c0b5440 con 0x7f717c0d2d70 2024-04-28T12:43:12.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.163+0000 7f7181e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f717c0b5d70 con 0x7f717c0d31d0 2024-04-28T12:43:12.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.163+0000 7f717aa00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f717c0d3780 0x7f717c0d3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33142/0 (socket says 172.21.15.131:33142) 2024-04-28T12:43:12.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.163+0000 7f717aa00640 1 -- 172.21.15.131:0/2157576237 learned_addr learned my addr 172.21.15.131:0/2157576237 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 599939979 0 0) 0x7f717c0b5160 con 0x7f717c0d3780 2024-04-28T12:43:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7160003520 con 0x7f717c0d3780 2024-04-28T12:43:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717b400640 1 --2- 172.21.15.131:0/2157576237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717c0d31d0 0x7f717c0d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 779254921 0 0) 0x7f7160003520 con 0x7f717c0d3780 2024-04-28T12:43:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717c0d31d0 msgr2=0x7f717c0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 --2- 172.21.15.131:0/2157576237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717c0d31d0 0x7f717c0d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 >> v1:172.21.15.157:6789/0 conn(0x7f717c0d2d70 legacy=0x7f717c0cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c0b5720 con 0x7f717c0d3780 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f716c001670 con 0x7f717c0d3780 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 >> v1:172.21.15.131:6789/0 conn(0x7f717c0d3780 legacy=0x7f717c0d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 --2- 172.21.15.131:0/2157576237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717c0d31d0 0x7f71600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 --2- 172.21.15.131:0/2157576237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7160003fb0 0x7f7160004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 --2- 172.21.15.131:0/2157576237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7160004d40 0x7f71600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f717c0b5d70 con 0x7f7160003fb0 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f716000bb00 con 0x7f7160004d40 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.164+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f716000bd00 con 0x7f717c0d31d0 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.165+0000 7f7173e00640 1 --2- 172.21.15.131:0/2157576237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7160003fb0 0x7f7160004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.165+0000 7f717aa00640 1 --2- 172.21.15.131:0/2157576237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717c0d31d0 0x7f71600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.165+0000 7f717b400640 1 --2- 172.21.15.131:0/2157576237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7160004d40 0x7f71600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:12.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.165+0000 7f7173e00640 1 -- 172.21.15.131:0/2157576237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717c0d31d0 msgr2=0x7f71600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:12.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.165+0000 7f7173e00640 1 --2- 172.21.15.131:0/2157576237 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f717c0d31d0 0x7f71600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:12.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.165+0000 7f7173e00640 1 -- 172.21.15.131:0/2157576237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7160004d40 msgr2=0x7f71600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:12.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.165+0000 7f7173e00640 1 --2- 172.21.15.131:0/2157576237 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7160004d40 0x7f71600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:12.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.165+0000 7f7173e00640 1 -- 172.21.15.131:0/2157576237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7160003520 con 0x7f7160003fb0 2024-04-28T12:43:12.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.166+0000 7f7173e00640 1 --2- 172.21.15.131:0/2157576237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7160003fb0 0x7f7160004800 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7164001750 tx=0x7f7160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f1cabc3ea5d5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:12.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.166+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164001370 con 0x7f7160003fb0 2024-04-28T12:43:12.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.166+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71640052d0 con 0x7f7160003fb0 2024-04-28T12:43:12.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.166+0000 7f717a000640 1 -- 172.21.15.131:0/2157576237 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716402ab20 con 0x7f7160003fb0 2024-04-28T12:43:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.167+0000 7f7181e00640 1 -- 172.21.15.131:0/2157576237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7160003fb0 msgr2=0x7f7160004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.167+0000 7f7181e00640 1 --2- 172.21.15.131:0/2157576237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7160003fb0 0x7f7160004800 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7164001750 tx=0x7f7160005830 comp rx=0 tx=0).stop 2024-04-28T12:43:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.167+0000 7f717b400640 1 -- 172.21.15.131:0/2157576237 reap_dead start 2024-04-28T12:43:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.167+0000 7f7181e00640 1 -- 172.21.15.131:0/2157576237 shutdown_connections 2024-04-28T12:43:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.167+0000 7f7181e00640 1 -- 172.21.15.131:0/2157576237 >> 172.21.15.131:0/2157576237 conn(0x7f717c08ec50 msgr2=0x7f717c09e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.167+0000 7f7181e00640 1 -- 172.21.15.131:0/2157576237 shutdown_connections 2024-04-28T12:43:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.167+0000 7f7181e00640 1 -- 172.21.15.131:0/2157576237 wait complete. 2024-04-28T12:43:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.169+0000 7f7181e00640 1 Processor -- start 2024-04-28T12:43:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.169+0000 7f7181e00640 1 -- start start 2024-04-28T12:43:12.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.170+0000 7f7181e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7160003fb0 0x7f717c0cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:12.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.170+0000 7f7181e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f717c0ca590 0x7f717c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:12.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.170+0000 7f7181e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f717c1f8360 0x7f717c15def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:12.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.170+0000 7f7181e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f717c0b4920 con 0x7f717c1f8360 2024-04-28T12:43:12.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.170+0000 7f7181e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f717c15f5e0 con 0x7f717c0ca590 2024-04-28T12:43:12.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.170+0000 7f7181e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f717c0b4aa0 con 0x7f7160003fb0 2024-04-28T12:43:12.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.170+0000 7f717b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f717c1f8360 0x7f717c15def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:12.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.170+0000 7f717b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f717c1f8360 0x7f717c15def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33746/0 (socket says 172.21.15.131:33746) 2024-04-28T12:43:12.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.170+0000 7f717b400640 1 -- 172.21.15.131:0/349350814 learned_addr learned my addr 172.21.15.131:0/349350814 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:12.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f717aa00640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7160003fb0 0x7f717c0cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:12.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f7173e00640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f717c0ca590 0x7f717c0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:12.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f717b400640 1 -- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7160003fb0 msgr2=0x7f717c0cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:12.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f717b400640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7160003fb0 0x7f717c0cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:12.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f717b400640 1 -- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f717c0ca590 msgr2=0x7f717c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:12.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f717b400640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f717c0ca590 0x7f717c0cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:12.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f717b400640 1 -- 172.21.15.131:0/349350814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c0cf3c0 con 0x7f717c1f8360 2024-04-28T12:43:12.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f7173e00640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f717c0ca590 0x7f717c0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:12.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f717b400640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f717c1f8360 0x7f717c15def0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f717c08f510 tx=0x7f71740115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:12.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f7173400640 1 -- 172.21.15.131:0/349350814 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174011e10 con 0x7f717c1f8360 2024-04-28T12:43:12.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f7173400640 1 -- 172.21.15.131:0/349350814 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f717401b070 con 0x7f717c1f8360 2024-04-28T12:43:12.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f7173400640 1 -- 172.21.15.131:0/349350814 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174003990 con 0x7f717c1f8360 2024-04-28T12:43:12.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.171+0000 7f7181e00640 1 -- 172.21.15.131:0/349350814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c0cb6c0 con 0x7f717c1f8360 2024-04-28T12:43:12.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.172+0000 7f7181e00640 1 -- 172.21.15.131:0/349350814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c0c9ab0 con 0x7f717c1f8360 2024-04-28T12:43:12.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.173+0000 7f7181e00640 1 -- 172.21.15.131:0/349350814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f717c069350 con 0x7f717c1f8360 2024-04-28T12:43:12.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.174+0000 7f7173400640 1 -- 172.21.15.131:0/349350814 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7174005d10 con 0x7f717c1f8360 2024-04-28T12:43:12.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.175+0000 7f7173400640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f715806c8f0 0x7f715806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:12.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.175+0000 7f7173400640 1 -- 172.21.15.131:0/349350814 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7f717408c370 con 0x7f717c1f8360 2024-04-28T12:43:12.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.175+0000 7f717aa00640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f715806c8f0 0x7f715806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:12.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.176+0000 7f717aa00640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f715806c8f0 0x7f715806edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f716c001050 tx=0x7f716c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:12.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.179+0000 7f7173400640 1 -- 172.21.15.131:0/349350814 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7174025020 con 0x7f717c1f8360 2024-04-28T12:43:12.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.420+0000 7f7181e00640 1 -- 172.21.15.131:0/349350814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f717c068c80 con 0x7f717c1f8360 2024-04-28T12:43:12.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.941+0000 7f7173400640 1 -- 172.21.15.131:0/349350814 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f717c0cf3c0 con 0x7f717c1f8360 2024-04-28T12:43:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.945+0000 7f7181e00640 1 -- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f715806c8f0 msgr2=0x7f715806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.945+0000 7f7181e00640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f715806c8f0 0x7f715806edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f716c001050 tx=0x7f716c009040 comp rx=0 tx=0).stop 2024-04-28T12:43:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.945+0000 7f7181e00640 1 -- 172.21.15.131:0/349350814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f717c1f8360 msgr2=0x7f717c15def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.945+0000 7f7181e00640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f717c1f8360 0x7f717c15def0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f717c08f510 tx=0x7f71740115a0 comp rx=0 tx=0).stop 2024-04-28T12:43:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.945+0000 7f7181e00640 1 -- 172.21.15.131:0/349350814 shutdown_connections 2024-04-28T12:43:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.945+0000 7f7181e00640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f715806c8f0 0x7f715806edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.945+0000 7f7181e00640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f717c1f8360 0x7f717c15def0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.945+0000 7f7181e00640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f717c0ca590 0x7f717c0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.945+0000 7f7181e00640 1 --2- 172.21.15.131:0/349350814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7160003fb0 0x7f717c0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.945+0000 7f7181e00640 1 -- 172.21.15.131:0/349350814 >> 172.21.15.131:0/349350814 conn(0x7f717c08ec50 msgr2=0x7f717c0d1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.946+0000 7f7181e00640 1 -- 172.21.15.131:0/349350814 shutdown_connections 2024-04-28T12:43:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:12.946+0000 7f7181e00640 1 -- 172.21.15.131:0/349350814 wait complete. 2024-04-28T12:43:12.964 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-28T12:43:13.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.177+0000 7fe006a00640 1 Processor -- start 2024-04-28T12:43:13.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.177+0000 7fe006a00640 1 -- start start 2024-04-28T12:43:13.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.178+0000 7fe006a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe000145450 0x7fe00006f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:13.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.178+0000 7fe006a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe000160090 0x7fe0000b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:13.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.178+0000 7fe006a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0000738c0 con 0x7fe0000b5100 2024-04-28T12:43:13.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.178+0000 7fe006a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe0000b5570 con 0x7fe000145450 2024-04-28T12:43:13.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.178+0000 7fe006a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe000073e40 con 0x7fe000160090 2024-04-28T12:43:13.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.179+0000 7fe006000640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7fe0000b5100 0x7fe000071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.131:51620/0 (socket says 172.21.15.131:51620) 2024-04-28T12:43:13.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.179+0000 7fe006000640 1 -- 172.21.15.131:0/4003717059 learned_addr learned my addr 172.21.15.131:0/4003717059 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:13.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.179+0000 7fe004c00640 1 --2- 172.21.15.131:0/4003717059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe000160090 0x7fe0000b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:13.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.179+0000 7fe005600640 1 --2- 172.21.15.131:0/4003717059 >> v2:172.21.15.157:3300/0 conn(0x7fe000145450 0x7fe00006f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:13.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.180+0000 7fe005600640 1 -- 172.21.15.131:0/4003717059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe000160090 msgr2=0x7fe0000b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:13.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.180+0000 7fe005600640 1 --2- 172.21.15.131:0/4003717059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe000160090 0x7fe0000b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:13.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.180+0000 7fe005600640 1 -- 172.21.15.131:0/4003717059 >> v1:172.21.15.157:6789/0 conn(0x7fe0000b5100 legacy=0x7fe000071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:13.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.180+0000 7fe005600640 1 -- 172.21.15.131:0/4003717059 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe000073b60 con 0x7fe000145450 2024-04-28T12:43:13.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.182+0000 7fe005600640 1 --2- 172.21.15.131:0/4003717059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe000145450 0x7fe00006f6c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdff40097a0 tx=0x7fdff402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6c5e1f9d7d9ad0f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:13.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.182+0000 7fdfffe00640 1 -- 172.21.15.131:0/4003717059 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4037070 con 0x7fe000145450 2024-04-28T12:43:13.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.182+0000 7fdfffe00640 1 -- 172.21.15.131:0/4003717059 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff4032440 con 0x7fe000145450 2024-04-28T12:43:13.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.182+0000 7fdfffe00640 1 -- 172.21.15.131:0/4003717059 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4032810 con 0x7fe000145450 2024-04-28T12:43:13.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.183+0000 7fe006a00640 1 -- 172.21.15.131:0/4003717059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe000145450 msgr2=0x7fe00006f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:13.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.183+0000 7fe006a00640 1 --2- 172.21.15.131:0/4003717059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe000145450 0x7fe00006f6c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdff40097a0 tx=0x7fdff402a180 comp rx=0 tx=0).stop 2024-04-28T12:43:13.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.183+0000 7fe006a00640 1 -- 172.21.15.131:0/4003717059 shutdown_connections 2024-04-28T12:43:13.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.183+0000 7fe006a00640 1 --2- 172.21.15.131:0/4003717059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe000160090 0x7fe0000b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:13.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.183+0000 7fe006a00640 1 --2- 172.21.15.131:0/4003717059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe000145450 0x7fe00006f6c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:13.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.183+0000 7fe006a00640 1 -- 172.21.15.131:0/4003717059 >> 172.21.15.131:0/4003717059 conn(0x7fe0000b26b0 msgr2=0x7fe0000b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:13.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.183+0000 7fe006a00640 1 -- 172.21.15.131:0/4003717059 shutdown_connections 2024-04-28T12:43:13.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.183+0000 7fe006a00640 1 -- 172.21.15.131:0/4003717059 wait complete. 2024-04-28T12:43:13.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.185+0000 7fe006a00640 1 Processor -- start 2024-04-28T12:43:13.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.185+0000 7fe006a00640 1 -- start start 2024-04-28T12:43:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.186+0000 7fe006a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0000b5100 0x7fe0000cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.186+0000 7fe006a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe000145450 0x7fe00013b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.186+0000 7fe006a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe000160090 0x7fe00013cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.186+0000 7fe006a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe000160510 con 0x7fe000160090 2024-04-28T12:43:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.186+0000 7fe006a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe00006eec0 con 0x7fe000145450 2024-04-28T12:43:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.186+0000 7fe006a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe0000cd4e0 con 0x7fe0000b5100 2024-04-28T12:43:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.187+0000 7fe006000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe000160090 0x7fe00013cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.187+0000 7fe006000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe000160090 0x7fe00013cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33760/0 (socket says 172.21.15.131:33760) 2024-04-28T12:43:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.187+0000 7fe006000640 1 -- 172.21.15.131:0/2215746395 learned_addr learned my addr 172.21.15.131:0/2215746395 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.187+0000 7fe005600640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0000b5100 0x7fe0000cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.187+0000 7fe004c00640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe000145450 0x7fe00013b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.187+0000 7fe006000640 1 -- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0000b5100 msgr2=0x7fe0000cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:13.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.187+0000 7fe006000640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0000b5100 0x7fe0000cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:13.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.187+0000 7fe006000640 1 -- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe000145450 msgr2=0x7fe00013b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:13.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.187+0000 7fe006000640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe000145450 0x7fe00013b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:13.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.187+0000 7fe006000640 1 -- 172.21.15.131:0/2215746395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00013be00 con 0x7fe000160090 2024-04-28T12:43:13.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.188+0000 7fe006000640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe000160090 0x7fe00013cf40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdff800ca80 tx=0x7fdff800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.188+0000 7fdffd600640 1 -- 172.21.15.131:0/2215746395 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8017070 con 0x7fe000160090 2024-04-28T12:43:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.188+0000 7fdffd600640 1 -- 172.21.15.131:0/2215746395 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff800ac30 con 0x7fe000160090 2024-04-28T12:43:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.188+0000 7fdffd600640 1 -- 172.21.15.131:0/2215746395 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8012600 con 0x7fe000160090 2024-04-28T12:43:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.188+0000 7fe006a00640 1 -- 172.21.15.131:0/2215746395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00013c0e0 con 0x7fe000160090 2024-04-28T12:43:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.188+0000 7fe006a00640 1 -- 172.21.15.131:0/2215746395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00013c980 con 0x7fe000160090 2024-04-28T12:43:13.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.189+0000 7fe006a00640 1 -- 172.21.15.131:0/2215746395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe00013c4d0 con 0x7fe000160090 2024-04-28T12:43:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.190+0000 7fdffd600640 1 -- 172.21.15.131:0/2215746395 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdff801a070 con 0x7fe000160090 2024-04-28T12:43:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.191+0000 7fdffd600640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdfe006c8f0 0x7fdfe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:13.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.191+0000 7fdffd600640 1 -- 172.21.15.131:0/2215746395 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6216+0+0 (secure 0 0 0) 0x7fdff808b910 con 0x7fe000160090 2024-04-28T12:43:13.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.192+0000 7fe005600640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdfe006c8f0 0x7fdfe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:13.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.193+0000 7fe005600640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdfe006c8f0 0x7fdfe006edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdff4034000 tx=0x7fdff402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:13.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.196+0000 7fdffd600640 1 -- 172.21.15.131:0/2215746395 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdff800ae20 con 0x7fe000160090 2024-04-28T12:43:13.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.400+0000 7fe006a00640 1 -- 172.21.15.131:0/2215746395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe000069350 con 0x7fe000160090 2024-04-28T12:43:13.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.952+0000 7fdffd600640 1 -- 172.21.15.131:0/2215746395 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) ==== 181+0+0 (secure 0 0 0) 0x7fdff80594e0 con 0x7fe000160090 2024-04-28T12:43:13.949 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-04-28T12:43:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.958+0000 7fe006a00640 1 -- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdfe006c8f0 msgr2=0x7fdfe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.958+0000 7fe006a00640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdfe006c8f0 0x7fdfe006edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdff4034000 tx=0x7fdff402e000 comp rx=0 tx=0).stop 2024-04-28T12:43:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.958+0000 7fe006a00640 1 -- 172.21.15.131:0/2215746395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe000160090 msgr2=0x7fe00013cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.958+0000 7fe006a00640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe000160090 0x7fe00013cf40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdff800ca80 tx=0x7fdff800cf40 comp rx=0 tx=0).stop 2024-04-28T12:43:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.958+0000 7fe006a00640 1 -- 172.21.15.131:0/2215746395 shutdown_connections 2024-04-28T12:43:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.958+0000 7fe006a00640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdfe006c8f0 0x7fdfe006edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.958+0000 7fe006a00640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe000160090 0x7fe00013cf40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:13.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.958+0000 7fe006a00640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe000145450 0x7fe00013b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:13.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.958+0000 7fe006a00640 1 --2- 172.21.15.131:0/2215746395 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0000b5100 0x7fe0000cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:13.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.958+0000 7fe006a00640 1 -- 172.21.15.131:0/2215746395 >> 172.21.15.131:0/2215746395 conn(0x7fe0000b26b0 msgr2=0x7fe000072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:13.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.959+0000 7fe006a00640 1 -- 172.21.15.131:0/2215746395 shutdown_connections 2024-04-28T12:43:13.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:13.959+0000 7fe006a00640 1 -- 172.21.15.131:0/2215746395 wait complete. 2024-04-28T12:43:13.978 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-28T12:43:14.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.182+0000 7f2661400640 1 Processor -- start 2024-04-28T12:43:14.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.183+0000 7f2661400640 1 -- start start 2024-04-28T12:43:14.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.184+0000 7f2661400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f265c156290 0x7f265c06f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:14.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.184+0000 7f2661400640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f265c163420 0x7f265c06f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:14.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.184+0000 7f2661400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265c06dfa0 con 0x7f265c072ba0 2024-04-28T12:43:14.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.184+0000 7f2661400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f265c06fee0 con 0x7f265c156290 2024-04-28T12:43:14.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.184+0000 7f2661400640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f265c0b5df0 con 0x7f265c163420 2024-04-28T12:43:14.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.185+0000 7f265a800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f265c156290 0x7f265c06f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:14.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.185+0000 7f265a800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f265c156290 0x7f265c06f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33768/0 (socket says 172.21.15.131:33768) 2024-04-28T12:43:14.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.185+0000 7f265a800640 1 -- 172.21.15.131:0/562054336 learned_addr learned my addr 172.21.15.131:0/562054336 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:14.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.185+0000 7f265a800640 1 -- 172.21.15.131:0/562054336 >> v2:172.21.15.157:3300/0 conn(0x7f265c163420 msgr2=0x7f265c06f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:14.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.185+0000 7f265a800640 1 --2- 172.21.15.131:0/562054336 >> v2:172.21.15.157:3300/0 conn(0x7f265c163420 0x7f265c06f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:14.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.185+0000 7f265a800640 1 -- 172.21.15.131:0/562054336 >> v1:172.21.15.157:6789/0 conn(0x7f265c072ba0 legacy=0x7f265c162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:14.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.185+0000 7f265a800640 1 -- 172.21.15.131:0/562054336 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c070110 con 0x7f265c156290 2024-04-28T12:43:14.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.186+0000 7f265a800640 1 --2- 172.21.15.131:0/562054336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c156290 0x7f265c06f000 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f265c06c3f0 tx=0x7f264c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b279b439de811c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.187+0000 7f2659e00640 1 -- 172.21.15.131:0/562054336 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c037070 con 0x7f265c156290 2024-04-28T12:43:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.187+0000 7f2659e00640 1 -- 172.21.15.131:0/562054336 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264c032440 con 0x7f265c156290 2024-04-28T12:43:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.187+0000 7f2659e00640 1 -- 172.21.15.131:0/562054336 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c032810 con 0x7f265c156290 2024-04-28T12:43:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.187+0000 7f2661400640 1 -- 172.21.15.131:0/562054336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c156290 msgr2=0x7f265c06f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.187+0000 7f2661400640 1 --2- 172.21.15.131:0/562054336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c156290 0x7f265c06f000 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f265c06c3f0 tx=0x7f264c02a2c0 comp rx=0 tx=0).stop 2024-04-28T12:43:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.187+0000 7f2661400640 1 -- 172.21.15.131:0/562054336 shutdown_connections 2024-04-28T12:43:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.187+0000 7f2661400640 1 --2- 172.21.15.131:0/562054336 >> v2:172.21.15.157:3300/0 conn(0x7f265c163420 0x7f265c06f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.187+0000 7f2661400640 1 --2- 172.21.15.131:0/562054336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c156290 0x7f265c06f000 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.187+0000 7f2661400640 1 -- 172.21.15.131:0/562054336 >> 172.21.15.131:0/562054336 conn(0x7f265c0b2790 msgr2=0x7f265c0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.187+0000 7f2661400640 1 -- 172.21.15.131:0/562054336 shutdown_connections 2024-04-28T12:43:14.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.188+0000 7f2661400640 1 -- 172.21.15.131:0/562054336 wait complete. 2024-04-28T12:43:14.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.190+0000 7f2661400640 1 Processor -- start 2024-04-28T12:43:14.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.190+0000 7f2661400640 1 -- start start 2024-04-28T12:43:14.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.191+0000 7f2661400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f265c072ba0 0x7f265c1f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:14.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.191+0000 7f2661400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c156290 0x7f265c15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:14.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.191+0000 7f2661400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f265c163420 0x7f265c15b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:14.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.191+0000 7f2661400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f265c06e800 con 0x7f265c156290 2024-04-28T12:43:14.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.191+0000 7f2661400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f265c06e520 con 0x7f265c163420 2024-04-28T12:43:14.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.191+0000 7f2661400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f265c073000 con 0x7f265c072ba0 2024-04-28T12:43:14.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.191+0000 7f2651e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c156290 0x7f265c15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:14.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.192+0000 7f2651e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c156290 0x7f265c15d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33774/0 (socket says 172.21.15.131:33774) 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.192+0000 7f2651e00640 1 -- 172.21.15.131:0/1860512957 learned_addr learned my addr 172.21.15.131:0/1860512957 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.192+0000 7f265a800640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f265c072ba0 0x7f265c1f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.192+0000 7f265b200640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f265c163420 0x7f265c15b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.192+0000 7f2651e00640 1 -- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f265c072ba0 msgr2=0x7f265c1f8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.192+0000 7f2651e00640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f265c072ba0 0x7f265c1f8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.192+0000 7f2651e00640 1 -- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f265c163420 msgr2=0x7f265c15b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.192+0000 7f2651e00640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f265c163420 0x7f265c15b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.192+0000 7f2651e00640 1 -- 172.21.15.131:0/1860512957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c1f8bc0 con 0x7f265c156290 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.192+0000 7f265a800640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f265c072ba0 0x7f265c1f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.192+0000 7f265b200640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f265c163420 0x7f265c15b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.192+0000 7f2651e00640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c156290 0x7f265c15d500 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f264400c410 tx=0x7f264400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.193+0000 7f2653400640 1 -- 172.21.15.131:0/1860512957 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644017070 con 0x7f265c156290 2024-04-28T12:43:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.193+0000 7f2653400640 1 -- 172.21.15.131:0/1860512957 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26440035f0 con 0x7f265c156290 2024-04-28T12:43:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.193+0000 7f2653400640 1 -- 172.21.15.131:0/1860512957 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644003a20 con 0x7f265c156290 2024-04-28T12:43:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.193+0000 7f2661400640 1 -- 172.21.15.131:0/1860512957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c1f8ea0 con 0x7f265c156290 2024-04-28T12:43:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.193+0000 7f2661400640 1 -- 172.21.15.131:0/1860512957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c15bd10 con 0x7f265c156290 2024-04-28T12:43:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.194+0000 7f2661400640 1 -- 172.21.15.131:0/1860512957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265c069350 con 0x7f265c156290 2024-04-28T12:43:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.195+0000 7f2653400640 1 -- 172.21.15.131:0/1860512957 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f264401a070 con 0x7f265c156290 2024-04-28T12:43:14.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.196+0000 7f2653400640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f263806c8f0 0x7f263806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:14.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.196+0000 7f2653400640 1 -- 172.21.15.131:0/1860512957 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5778+0+0 (secure 0 0 0) 0x7f264408b060 con 0x7f265c156290 2024-04-28T12:43:14.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.196+0000 7f265a800640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f263806c8f0 0x7f263806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:14.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.198+0000 7f265a800640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f263806c8f0 0x7f263806edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f264c034000 tx=0x7f264c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:14.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.200+0000 7f2653400640 1 -- 172.21.15.131:0/1860512957 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2644059430 con 0x7f265c156290 2024-04-28T12:43:14.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.413+0000 7f2661400640 1 -- 172.21.15.131:0/1860512957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f265c0b3cd0 con 0x7f265c156290 2024-04-28T12:43:14.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.958+0000 7f2653400640 1 -- 172.21.15.131:0/1860512957 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) ==== 169+0+0 (secure 0 0 0) 0x7f2644058d30 con 0x7f265c156290 2024-04-28T12:43:14.955 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-04-28T12:43:14.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.962+0000 7f2661400640 1 -- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f263806c8f0 msgr2=0x7f263806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.962+0000 7f2661400640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f263806c8f0 0x7f263806edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f264c034000 tx=0x7f264c02e000 comp rx=0 tx=0).stop 2024-04-28T12:43:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.962+0000 7f2661400640 1 -- 172.21.15.131:0/1860512957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c156290 msgr2=0x7f265c15d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.962+0000 7f2661400640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c156290 0x7f265c15d500 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f264400c410 tx=0x7f264400c8d0 comp rx=0 tx=0).stop 2024-04-28T12:43:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.962+0000 7f2661400640 1 -- 172.21.15.131:0/1860512957 shutdown_connections 2024-04-28T12:43:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.962+0000 7f2661400640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f263806c8f0 0x7f263806edb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.962+0000 7f2661400640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f265c163420 0x7f265c15b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.962+0000 7f2661400640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c156290 0x7f265c15d500 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.962+0000 7f2661400640 1 --2- 172.21.15.131:0/1860512957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f265c072ba0 0x7f265c1f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.962+0000 7f2661400640 1 -- 172.21.15.131:0/1860512957 >> 172.21.15.131:0/1860512957 conn(0x7f265c0b2790 msgr2=0x7f265c071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:14.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.963+0000 7f2661400640 1 -- 172.21.15.131:0/1860512957 shutdown_connections 2024-04-28T12:43:14.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:14.964+0000 7f2661400640 1 -- 172.21.15.131:0/1860512957 wait complete. 2024-04-28T12:43:14.983 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-28T12:43:15.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.178+0000 7f5c34e00640 1 Processor -- start 2024-04-28T12:43:15.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.178+0000 7f5c34e00640 1 -- start start 2024-04-28T12:43:15.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.179+0000 7f5c34e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f5c30072a90 0x7f5c300cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:15.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.179+0000 7f5c34e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c3015d5e0 0x7f5c300cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:15.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.179+0000 7f5c34e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c30146160 con 0x7f5c30145d80 2024-04-28T12:43:15.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.179+0000 7f5c34e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f5c3006dfb0 con 0x7f5c30072a90 2024-04-28T12:43:15.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.179+0000 7f5c34e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5c30074180 con 0x7f5c3015d5e0 2024-04-28T12:43:15.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.180+0000 7f5c2e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c3015d5e0 0x7f5c300cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:15.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.180+0000 7f5c2de00640 1 --1- >> v1:172.21.15.157:6789/0 conn(0x7f5c30145d80 0x7f5c30072380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.157:6789/0 says I am v1:172.21.15.131:51652/0 (socket says 172.21.15.131:51652) 2024-04-28T12:43:15.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.180+0000 7f5c2de00640 1 -- 172.21.15.131:0/198479012 learned_addr learned my addr 172.21.15.131:0/198479012 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:15.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.180+0000 7f5c2e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c3015d5e0 0x7f5c300cf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:58392/0 (socket says 172.21.15.131:58392) 2024-04-28T12:43:15.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.180+0000 7f5c2d400640 1 --2- 172.21.15.131:0/198479012 >> v2:172.21.15.157:3300/0 conn(0x7f5c30072a90 0x7f5c300cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:15.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.181+0000 7f5c2ca00640 1 -- 172.21.15.131:0/198479012 <== mon.1 v1:172.21.15.157:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 424202901 0 0) 0x7f5c30146160 con 0x7f5c30145d80 2024-04-28T12:43:15.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.181+0000 7f5c2ca00640 1 -- 172.21.15.131:0/198479012 --> v1:172.21.15.157:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c14003520 con 0x7f5c30145d80 2024-04-28T12:43:15.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.181+0000 7f5c2d400640 1 -- 172.21.15.131:0/198479012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c3015d5e0 msgr2=0x7f5c300cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:15.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.181+0000 7f5c2d400640 1 --2- 172.21.15.131:0/198479012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c3015d5e0 0x7f5c300cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.181+0000 7f5c2d400640 1 -- 172.21.15.131:0/198479012 >> v1:172.21.15.157:6789/0 conn(0x7f5c30145d80 legacy=0x7f5c30072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:15.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.181+0000 7f5c2d400640 1 -- 172.21.15.131:0/198479012 --> v2:172.21.15.157:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c300d0710 con 0x7f5c30072a90 2024-04-28T12:43:15.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.181+0000 7f5c2e800640 1 --2- 172.21.15.131:0/198479012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c3015d5e0 0x7f5c300cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:43:15.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.183+0000 7f5c2d400640 1 --2- 172.21.15.131:0/198479012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30072a90 0x7f5c300cf460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0097a0 tx=0x7f5c1c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff6c4fcbc2284d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:15.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.183+0000 7f5c2ca00640 1 -- 172.21.15.131:0/198479012 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c037070 con 0x7f5c30072a90 2024-04-28T12:43:15.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.183+0000 7f5c2ca00640 1 -- 172.21.15.131:0/198479012 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c1c032440 con 0x7f5c30072a90 2024-04-28T12:43:15.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.183+0000 7f5c2ca00640 1 -- 172.21.15.131:0/198479012 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c0327f0 con 0x7f5c30072a90 2024-04-28T12:43:15.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.184+0000 7f5c34e00640 1 -- 172.21.15.131:0/198479012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30072a90 msgr2=0x7f5c300cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:15.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.184+0000 7f5c34e00640 1 --2- 172.21.15.131:0/198479012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30072a90 0x7f5c300cf460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0097a0 tx=0x7f5c1c02a180 comp rx=0 tx=0).stop 2024-04-28T12:43:15.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.184+0000 7f5c34e00640 1 -- 172.21.15.131:0/198479012 shutdown_connections 2024-04-28T12:43:15.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.184+0000 7f5c34e00640 1 --2- 172.21.15.131:0/198479012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c3015d5e0 0x7f5c300cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.184+0000 7f5c34e00640 1 --2- 172.21.15.131:0/198479012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30072a90 0x7f5c300cf460 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.184+0000 7f5c34e00640 1 -- 172.21.15.131:0/198479012 >> 172.21.15.131:0/198479012 conn(0x7f5c300b2900 msgr2=0x7f5c300b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:15.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.184+0000 7f5c34e00640 1 -- 172.21.15.131:0/198479012 shutdown_connections 2024-04-28T12:43:15.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.184+0000 7f5c34e00640 1 -- 172.21.15.131:0/198479012 wait complete. 2024-04-28T12:43:15.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.185+0000 7f5c34e00640 1 Processor -- start 2024-04-28T12:43:15.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.186+0000 7f5c34e00640 1 -- start start 2024-04-28T12:43:15.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.186+0000 7f5c34e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c30072a90 0x7f5c3015c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:15.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.186+0000 7f5c34e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30145d80 0x7f5c3013ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:15.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.186+0000 7f5c34e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c3015d5e0 0x7f5c301f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:15.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.186+0000 7f5c34e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5c300d09f0 con 0x7f5c3015d5e0 2024-04-28T12:43:15.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.186+0000 7f5c34e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3006e530 con 0x7f5c30145d80 2024-04-28T12:43:15.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.186+0000 7f5c34e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5c301468c0 con 0x7f5c30072a90 2024-04-28T12:43:15.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.187+0000 7f5c2e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c3015d5e0 0x7f5c301f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:15.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.187+0000 7f5c2e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c3015d5e0 0x7f5c301f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33776/0 (socket says 172.21.15.131:33776) 2024-04-28T12:43:15.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.187+0000 7f5c2e800640 1 -- 172.21.15.131:0/2743582017 learned_addr learned my addr 172.21.15.131:0/2743582017 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:15.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.187+0000 7f5c2de00640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c30072a90 0x7f5c3015c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:15.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.187+0000 7f5c2d400640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30145d80 0x7f5c3013ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.187+0000 7f5c2e800640 1 -- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c30072a90 msgr2=0x7f5c3015c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.187+0000 7f5c2e800640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c30072a90 0x7f5c3015c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.187+0000 7f5c2e800640 1 -- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30145d80 msgr2=0x7f5c3013ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.187+0000 7f5c2e800640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30145d80 0x7f5c3013ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.187+0000 7f5c2e800640 1 -- 172.21.15.131:0/2743582017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3013e5a0 con 0x7f5c3015d5e0 2024-04-28T12:43:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.187+0000 7f5c2d400640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30145d80 0x7f5c3013ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.188+0000 7f5c2e800640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c3015d5e0 0x7f5c301f4b00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009e10 tx=0x7f5c2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.188+0000 7f5c26000640 1 -- 172.21.15.131:0/2743582017 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28018020 con 0x7f5c3015d5e0 2024-04-28T12:43:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.188+0000 7f5c26000640 1 -- 172.21.15.131:0/2743582017 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c28014480 con 0x7f5c3015d5e0 2024-04-28T12:43:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.188+0000 7f5c26000640 1 -- 172.21.15.131:0/2743582017 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28014850 con 0x7f5c3015d5e0 2024-04-28T12:43:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.188+0000 7f5c34e00640 1 -- 172.21.15.131:0/2743582017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3013e820 con 0x7f5c3015d5e0 2024-04-28T12:43:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.188+0000 7f5c34e00640 1 -- 172.21.15.131:0/2743582017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c301f58a0 con 0x7f5c3015d5e0 2024-04-28T12:43:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.189+0000 7f5c34e00640 1 -- 172.21.15.131:0/2743582017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c30069350 con 0x7f5c3015d5e0 2024-04-28T12:43:15.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.190+0000 7f5c26000640 1 -- 172.21.15.131:0/2743582017 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c28005d10 con 0x7f5c3015d5e0 2024-04-28T12:43:15.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.191+0000 7f5c26000640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5c1006c8f0 0x7f5c1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:15.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.191+0000 7f5c26000640 1 -- 172.21.15.131:0/2743582017 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f5c2808c7a0 con 0x7f5c3015d5e0 2024-04-28T12:43:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.192+0000 7f5c2de00640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5c1006c8f0 0x7f5c1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:15.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.193+0000 7f5c2de00640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5c1006c8f0 0x7f5c1006edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5c20005880 tx=0x7f5c20009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.196+0000 7f5c26000640 1 -- 172.21.15.131:0/2743582017 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c28022030 con 0x7f5c3015d5e0 2024-04-28T12:43:15.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.401+0000 7f5c34e00640 1 -- 172.21.15.131:0/2743582017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5c301f5a70 con 0x7f5c3015d5e0 2024-04-28T12:43:15.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.409+0000 7f5c26000640 1 -- 172.21.15.131:0/2743582017 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f5c3013e5a0 con 0x7f5c3015d5e0 2024-04-28T12:43:15.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.411+0000 7f5c34e00640 1 -- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5c1006c8f0 msgr2=0x7f5c1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:15.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.411+0000 7f5c34e00640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5c1006c8f0 0x7f5c1006edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5c20005880 tx=0x7f5c20009040 comp rx=0 tx=0).stop 2024-04-28T12:43:15.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.411+0000 7f5c34e00640 1 -- 172.21.15.131:0/2743582017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c3015d5e0 msgr2=0x7f5c301f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:15.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.411+0000 7f5c34e00640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c3015d5e0 0x7f5c301f4b00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009e10 tx=0x7f5c2800c490 comp rx=0 tx=0).stop 2024-04-28T12:43:15.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.412+0000 7f5c34e00640 1 -- 172.21.15.131:0/2743582017 shutdown_connections 2024-04-28T12:43:15.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.412+0000 7f5c34e00640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f5c1006c8f0 0x7f5c1006edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.412+0000 7f5c34e00640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c3015d5e0 0x7f5c301f4b00 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.412+0000 7f5c34e00640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c30145d80 0x7f5c3013ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.412+0000 7f5c34e00640 1 --2- 172.21.15.131:0/2743582017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c30072a90 0x7f5c3015c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.412+0000 7f5c34e00640 1 -- 172.21.15.131:0/2743582017 >> 172.21.15.131:0/2743582017 conn(0x7f5c300b2900 msgr2=0x7f5c300b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.412+0000 7f5c34e00640 1 -- 172.21.15.131:0/2743582017 shutdown_connections 2024-04-28T12:43:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.412+0000 7f5c34e00640 1 -- 172.21.15.131:0/2743582017 wait complete. 2024-04-28T12:43:15.427 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-04-28T12:43:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.626+0000 7f8ae6800640 1 Processor -- start 2024-04-28T12:43:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.626+0000 7f8ae6800640 1 -- start start 2024-04-28T12:43:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.628+0000 7f8ae6800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ae015bb90 0x7f8ae00737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.628+0000 7f8ae6800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f8ae00af710 0x7f8ae00afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.628+0000 7f8ae6800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ae0072db0 con 0x7f8ae015f610 2024-04-28T12:43:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.628+0000 7f8ae6800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae00cec10 con 0x7f8ae00af710 2024-04-28T12:43:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.628+0000 7f8ae6800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae00b00b0 con 0x7f8ae015bb90 2024-04-28T12:43:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.628+0000 7f8ae5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ae015bb90 0x7f8ae00737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.629+0000 7f8ae5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ae015bb90 0x7f8ae00737a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:58406/0 (socket says 172.21.15.131:58406) 2024-04-28T12:43:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.629+0000 7f8ae5400640 1 -- 172.21.15.131:0/3092959404 learned_addr learned my addr 172.21.15.131:0/3092959404 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.629+0000 7f8ae4a00640 1 --2- 172.21.15.131:0/3092959404 >> v2:172.21.15.157:3300/0 conn(0x7f8ae00af710 0x7f8ae00afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.629+0000 7f8ae5400640 1 -- 172.21.15.131:0/3092959404 >> v2:172.21.15.157:3300/0 conn(0x7f8ae00af710 msgr2=0x7f8ae00afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.629+0000 7f8ae5400640 1 --2- 172.21.15.131:0/3092959404 >> v2:172.21.15.157:3300/0 conn(0x7f8ae00af710 0x7f8ae00afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.629+0000 7f8ae5400640 1 -- 172.21.15.131:0/3092959404 >> v1:172.21.15.157:6789/0 conn(0x7f8ae015f610 legacy=0x7f8ae015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.629+0000 7f8ae5400640 1 -- 172.21.15.131:0/3092959404 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae00cee40 con 0x7f8ae015bb90 2024-04-28T12:43:15.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.631+0000 7f8ae5400640 1 --2- 172.21.15.131:0/3092959404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ae015bb90 0x7f8ae00737a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009870 tx=0x7f8ad402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99a153eff6987c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:15.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.631+0000 7f8adfe00640 1 -- 172.21.15.131:0/3092959404 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4037070 con 0x7f8ae015bb90 2024-04-28T12:43:15.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.631+0000 7f8adfe00640 1 -- 172.21.15.131:0/3092959404 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad4032440 con 0x7f8ae015bb90 2024-04-28T12:43:15.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.632+0000 7f8adfe00640 1 -- 172.21.15.131:0/3092959404 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4032810 con 0x7f8ae015bb90 2024-04-28T12:43:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.632+0000 7f8ae6800640 1 -- 172.21.15.131:0/3092959404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ae015bb90 msgr2=0x7f8ae00737a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.632+0000 7f8ae6800640 1 --2- 172.21.15.131:0/3092959404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ae015bb90 0x7f8ae00737a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009870 tx=0x7f8ad402a270 comp rx=0 tx=0).stop 2024-04-28T12:43:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.632+0000 7f8ae6800640 1 -- 172.21.15.131:0/3092959404 shutdown_connections 2024-04-28T12:43:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.632+0000 7f8ae6800640 1 --2- 172.21.15.131:0/3092959404 >> v2:172.21.15.157:3300/0 conn(0x7f8ae00af710 0x7f8ae00afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.632+0000 7f8ae6800640 1 --2- 172.21.15.131:0/3092959404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ae015bb90 0x7f8ae00737a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.632+0000 7f8ae6800640 1 -- 172.21.15.131:0/3092959404 >> 172.21.15.131:0/3092959404 conn(0x7f8ae0068090 msgr2=0x7f8ae0068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.632+0000 7f8ae6800640 1 -- 172.21.15.131:0/3092959404 shutdown_connections 2024-04-28T12:43:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.632+0000 7f8ae6800640 1 -- 172.21.15.131:0/3092959404 wait complete. 2024-04-28T12:43:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.634+0000 7f8ae6800640 1 Processor -- start 2024-04-28T12:43:15.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.634+0000 7f8ae6800640 1 -- start start 2024-04-28T12:43:15.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.635+0000 7f8ae6800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ae00af710 0x7f8ae01f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.635+0000 7f8ae6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ae015bb90 0x7f8ae00ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.635+0000 7f8ae6800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ae015f610 0x7f8ae00aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.635+0000 7f8ae6800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae00d0610 con 0x7f8ae015bb90 2024-04-28T12:43:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.635+0000 7f8ae6800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae00725a0 con 0x7f8ae015f610 2024-04-28T12:43:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.635+0000 7f8ae6800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae0072ad0 con 0x7f8ae00af710 2024-04-28T12:43:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.635+0000 7f8ae4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ae015bb90 0x7f8ae00ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.635+0000 7f8ae4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ae015bb90 0x7f8ae00ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33782/0 (socket says 172.21.15.131:33782) 2024-04-28T12:43:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.635+0000 7f8ae4a00640 1 -- 172.21.15.131:0/1516855085 learned_addr learned my addr 172.21.15.131:0/1516855085 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:15.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.636+0000 7f8ae4a00640 1 -- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ae00af710 msgr2=0x7f8ae01f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:15.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.636+0000 7f8ae4a00640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ae00af710 0x7f8ae01f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.636+0000 7f8ae4a00640 1 -- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ae015f610 msgr2=0x7f8ae00aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.636+0000 7f8ae5e00640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ae015f610 0x7f8ae00aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.636+0000 7f8ae4a00640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ae015f610 0x7f8ae00aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.636+0000 7f8ae4a00640 1 -- 172.21.15.131:0/1516855085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae01f8df0 con 0x7f8ae015bb90 2024-04-28T12:43:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.636+0000 7f8ae5e00640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ae015f610 0x7f8ae00aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.636+0000 7f8ae4a00640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ae015bb90 0x7f8ae00ae8c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00e9b0 tx=0x7f8acc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.637+0000 7f8add600640 1 -- 172.21.15.131:0/1516855085 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc017070 con 0x7f8ae015bb90 2024-04-28T12:43:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.637+0000 7f8add600640 1 -- 172.21.15.131:0/1516855085 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8acc00c9a0 con 0x7f8ae015bb90 2024-04-28T12:43:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.637+0000 7f8add600640 1 -- 172.21.15.131:0/1516855085 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00cd90 con 0x7f8ae015bb90 2024-04-28T12:43:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.637+0000 7f8ae6800640 1 -- 172.21.15.131:0/1516855085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae01f90d0 con 0x7f8ae015bb90 2024-04-28T12:43:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.637+0000 7f8ae6800640 1 -- 172.21.15.131:0/1516855085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae01f9690 con 0x7f8ae015bb90 2024-04-28T12:43:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.638+0000 7f8ae6800640 1 -- 172.21.15.131:0/1516855085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae0069350 con 0x7f8ae015bb90 2024-04-28T12:43:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.639+0000 7f8add600640 1 -- 172.21.15.131:0/1516855085 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8acc01a070 con 0x7f8ae015bb90 2024-04-28T12:43:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.640+0000 7f8add600640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8ac006c920 0x7f8ac006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.640+0000 7f8add600640 1 -- 172.21.15.131:0/1516855085 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f8acc08be10 con 0x7f8ae015bb90 2024-04-28T12:43:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.641+0000 7f8ae5400640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8ac006c920 0x7f8ac006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:15.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.642+0000 7f8ae5400640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8ac006c920 0x7f8ac006ede0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4034000 tx=0x7f8ad402c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:15.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.645+0000 7f8add600640 1 -- 172.21.15.131:0/1516855085 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8acc08f030 con 0x7f8ae015bb90 2024-04-28T12:43:15.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.919+0000 7f8ae6800640 1 -- 172.21.15.131:0/1516855085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) -- 0x7f8ae015f380 con 0x7f8ae015bb90 2024-04-28T12:43:15.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.970+0000 7f8add600640 1 -- 172.21.15.131:0/1516855085 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v256) ==== 120+0+0 (secure 0 0 0) 0x7f8acc05a310 con 0x7f8ae015bb90 2024-04-28T12:43:15.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.973+0000 7f8ae6800640 1 -- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8ac006c920 msgr2=0x7f8ac006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:15.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.973+0000 7f8ae6800640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8ac006c920 0x7f8ac006ede0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4034000 tx=0x7f8ad402c000 comp rx=0 tx=0).stop 2024-04-28T12:43:15.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.973+0000 7f8ae6800640 1 -- 172.21.15.131:0/1516855085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ae015bb90 msgr2=0x7f8ae00ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:15.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.973+0000 7f8ae6800640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ae015bb90 0x7f8ae00ae8c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00e9b0 tx=0x7f8acc00ee70 comp rx=0 tx=0).stop 2024-04-28T12:43:15.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.973+0000 7f8ae6800640 1 -- 172.21.15.131:0/1516855085 shutdown_connections 2024-04-28T12:43:15.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.973+0000 7f8ae6800640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f8ac006c920 0x7f8ac006ede0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.973+0000 7f8ae6800640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8ae015f610 0x7f8ae00aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.973+0000 7f8ae6800640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ae015bb90 0x7f8ae00ae8c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.973+0000 7f8ae6800640 1 --2- 172.21.15.131:0/1516855085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8ae00af710 0x7f8ae01f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:15.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.973+0000 7f8ae6800640 1 -- 172.21.15.131:0/1516855085 >> 172.21.15.131:0/1516855085 conn(0x7f8ae0068090 msgr2=0x7f8ae00b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:15.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.973+0000 7f8ae6800640 1 -- 172.21.15.131:0/1516855085 shutdown_connections 2024-04-28T12:43:15.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:15.973+0000 7f8ae6800640 1 -- 172.21.15.131:0/1516855085 wait complete. 2024-04-28T12:43:15.989 INFO:tasks.cephfs_test_runner:test_acls (tasks.cephfs.test_acls.TestACLs) ... ok 2024-04-28T12:43:15.989 INFO:tasks.cephfs_test_runner: 2024-04-28T12:43:15.989 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-28T12:43:15.989 INFO:tasks.cephfs_test_runner:Ran 1 test in 289.742s 2024-04-28T12:43:15.989 INFO:tasks.cephfs_test_runner: 2024-04-28T12:43:15.989 INFO:tasks.cephfs_test_runner:OK 2024-04-28T12:43:15.990 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-28T12:43:15.999 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-28T12:43:16.042 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-28T12:43:16.043 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:43:16.043 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:43:16.092 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-28T12:43:16.093 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-04-28T12:43:16.093 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:43:16.190 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-28T12:43:16.190 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:43:16.190 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:43:16.248 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-28T12:43:16.248 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-28T12:43:16.248 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:43:16.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-28T12:43:16.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:43:16.301 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-28T12:43:16.354 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-28T12:43:16.355 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-28T12:43:16.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:43:16.356 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-28T12:43:16.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:43:16.444 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-28T12:43:16.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:43:16.515 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-04-28T12:43:16.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:43:16.637 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-04-28T12:43:16.691 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-28T12:43:16.691 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-28T12:43:16.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:43:16.692 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:43:16.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:43:16.777 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-28T12:43:16.834 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:43:16.838 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:43:16.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:43:16.850 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-04-28T12:43:16.969 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-28T12:43:16.979 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-28T12:43:16.979 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:43:17.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.193+0000 7fe0cf200640 1 Processor -- start 2024-04-28T12:43:17.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.193+0000 7fe0cf200640 1 -- start start 2024-04-28T12:43:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.194+0000 7fe0cf200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe0c8072b70 0x7fe0c80d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.194+0000 7fe0cf200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe0c80d1060 0x7fe0c806dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.194+0000 7fe0cf200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0c8146ba0 con 0x7fe0c81467c0 2024-04-28T12:43:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.194+0000 7fe0cf200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c8073dd0 con 0x7fe0c8072b70 2024-04-28T12:43:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.194+0000 7fe0cf200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c80731c0 con 0x7fe0c80d1060 2024-04-28T12:43:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.195+0000 7fe0c7e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe0c81467c0 0x7fe0c80b3ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33156/0 (socket says 172.21.15.131:33156) 2024-04-28T12:43:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.195+0000 7fe0c7e00640 1 -- 172.21.15.131:0/1594049116 learned_addr learned my addr 172.21.15.131:0/1594049116 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.195+0000 7fe0bf400640 1 --2- 172.21.15.131:0/1594049116 >> v2:172.21.15.131:3300/0 conn(0x7fe0c8072b70 0x7fe0c80d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.195+0000 7fe0cd200640 1 --2- 172.21.15.131:0/1594049116 >> v2:172.21.15.157:3300/0 conn(0x7fe0c80d1060 0x7fe0c806dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.195+0000 7fe0c7400640 1 -- 172.21.15.131:0/1594049116 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4098243699 0 0) 0x7fe0c8146ba0 con 0x7fe0c81467c0 2024-04-28T12:43:17.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.195+0000 7fe0c7400640 1 -- 172.21.15.131:0/1594049116 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0ac003520 con 0x7fe0c81467c0 2024-04-28T12:43:17.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.195+0000 7fe0bf400640 1 -- 172.21.15.131:0/1594049116 >> v2:172.21.15.157:3300/0 conn(0x7fe0c80d1060 msgr2=0x7fe0c806dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:17.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.195+0000 7fe0bf400640 1 --2- 172.21.15.131:0/1594049116 >> v2:172.21.15.157:3300/0 conn(0x7fe0c80d1060 0x7fe0c806dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.195+0000 7fe0bf400640 1 -- 172.21.15.131:0/1594049116 >> v1:172.21.15.131:6789/0 conn(0x7fe0c81467c0 legacy=0x7fe0c80b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.196+0000 7fe0bf400640 1 -- 172.21.15.131:0/1594049116 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c80734a0 con 0x7fe0c8072b70 2024-04-28T12:43:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.197+0000 7fe0bf400640 1 --2- 172.21.15.131:0/1594049116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c8072b70 0x7fe0c80d0b20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80720f0 tx=0x7fe0b002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1c8a7ec51fc7b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.197+0000 7fe0c7400640 1 -- 172.21.15.131:0/1594049116 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0037070 con 0x7fe0c8072b70 2024-04-28T12:43:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.197+0000 7fe0c7400640 1 -- 172.21.15.131:0/1594049116 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b0032440 con 0x7fe0c8072b70 2024-04-28T12:43:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.197+0000 7fe0c7400640 1 -- 172.21.15.131:0/1594049116 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00327f0 con 0x7fe0c8072b70 2024-04-28T12:43:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.198+0000 7fe0cf200640 1 -- 172.21.15.131:0/1594049116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c8072b70 msgr2=0x7fe0c80d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.198+0000 7fe0cf200640 1 --2- 172.21.15.131:0/1594049116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c8072b70 0x7fe0c80d0b20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80720f0 tx=0x7fe0b002a1a0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.198+0000 7fe0cf200640 1 -- 172.21.15.131:0/1594049116 shutdown_connections 2024-04-28T12:43:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.198+0000 7fe0cf200640 1 --2- 172.21.15.131:0/1594049116 >> v2:172.21.15.157:3300/0 conn(0x7fe0c80d1060 0x7fe0c806dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.198+0000 7fe0cf200640 1 --2- 172.21.15.131:0/1594049116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c8072b70 0x7fe0c80d0b20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.198+0000 7fe0cf200640 1 -- 172.21.15.131:0/1594049116 >> 172.21.15.131:0/1594049116 conn(0x7fe0c80b2900 msgr2=0x7fe0c80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:17.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.198+0000 7fe0cf200640 1 -- 172.21.15.131:0/1594049116 shutdown_connections 2024-04-28T12:43:17.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.198+0000 7fe0cf200640 1 -- 172.21.15.131:0/1594049116 wait complete. 2024-04-28T12:43:17.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.200+0000 7fe0cf200640 1 Processor -- start 2024-04-28T12:43:17.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.200+0000 7fe0cf200640 1 -- start start 2024-04-28T12:43:17.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.201+0000 7fe0cf200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c8072b70 0x7fe0c815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.201+0000 7fe0cf200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0c80d1060 0x7fe0c813ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.201+0000 7fe0cf200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c81467c0 0x7fe0c815c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.201+0000 7fe0cf200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c806e530 con 0x7fe0c81467c0 2024-04-28T12:43:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.201+0000 7fe0cf200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c8073a60 con 0x7fe0c80d1060 2024-04-28T12:43:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.201+0000 7fe0cf200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c806e6b0 con 0x7fe0c8072b70 2024-04-28T12:43:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.201+0000 7fe0cd200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c81467c0 0x7fe0c815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.201+0000 7fe0cd200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c81467c0 0x7fe0c815c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33812/0 (socket says 172.21.15.131:33812) 2024-04-28T12:43:17.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.201+0000 7fe0cd200640 1 -- 172.21.15.131:0/2816238805 learned_addr learned my addr 172.21.15.131:0/2816238805 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.201+0000 7fe0c7e00640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c8072b70 0x7fe0c815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0bf400640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0c80d1060 0x7fe0c813ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0cd200640 1 -- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c8072b70 msgr2=0x7fe0c815d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0cd200640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c8072b70 0x7fe0c815d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0cd200640 1 -- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0c80d1060 msgr2=0x7fe0c813ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0cd200640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0c80d1060 0x7fe0c813ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0cd200640 1 -- 172.21.15.131:0/2816238805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c815dbe0 con 0x7fe0c81467c0 2024-04-28T12:43:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0bf400640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0c80d1060 0x7fe0c813ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0cd200640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c81467c0 0x7fe0c815c420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009dd0 tx=0x7fe0c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0c4c00640 1 -- 172.21.15.131:0/2816238805 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0018020 con 0x7fe0c81467c0 2024-04-28T12:43:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0c4c00640 1 -- 172.21.15.131:0/2816238805 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c0014480 con 0x7fe0c81467c0 2024-04-28T12:43:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0c4c00640 1 -- 172.21.15.131:0/2816238805 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0014870 con 0x7fe0c81467c0 2024-04-28T12:43:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0cf200640 1 -- 172.21.15.131:0/2816238805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c815dec0 con 0x7fe0c81467c0 2024-04-28T12:43:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.202+0000 7fe0cf200640 1 -- 172.21.15.131:0/2816238805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c815e480 con 0x7fe0c81467c0 2024-04-28T12:43:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.204+0000 7fe0cf200640 1 -- 172.21.15.131:0/2816238805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c8069350 con 0x7fe0c81467c0 2024-04-28T12:43:17.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.205+0000 7fe0c4c00640 1 -- 172.21.15.131:0/2816238805 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0c0005d10 con 0x7fe0c81467c0 2024-04-28T12:43:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.206+0000 7fe0c4c00640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe0a406c8f0 0x7fe0a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.206+0000 7fe0c4c00640 1 -- 172.21.15.131:0/2816238805 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fe0c008c550 con 0x7fe0c81467c0 2024-04-28T12:43:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.206+0000 7fe0c7e00640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe0a406c8f0 0x7fe0a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.207+0000 7fe0c7e00640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe0a406c8f0 0x7fe0a406edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80d02c0 tx=0x7fe0b8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:17.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.211+0000 7fe0c4c00640 1 -- 172.21.15.131:0/2816238805 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe0c0022070 con 0x7fe0c81467c0 2024-04-28T12:43:17.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.404+0000 7fe0cf200640 1 -- 172.21.15.131:0/2816238805 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0c80b53a0 con 0x7fe0a406c8f0 2024-04-28T12:43:17.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.407+0000 7fe0c4c00640 1 -- 172.21.15.131:0/2816238805 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13489 (secure 0 0 0) 0x7fe0c80b53a0 con 0x7fe0a406c8f0 2024-04-28T12:43:17.404 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:17.404 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:43:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.409+0000 7fe0cf200640 1 -- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe0a406c8f0 msgr2=0x7fe0a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.409+0000 7fe0cf200640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe0a406c8f0 0x7fe0a406edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80d02c0 tx=0x7fe0b8009040 comp rx=0 tx=0).stop 2024-04-28T12:43:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.409+0000 7fe0cf200640 1 -- 172.21.15.131:0/2816238805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c81467c0 msgr2=0x7fe0c815c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.409+0000 7fe0cf200640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c81467c0 0x7fe0c815c420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009dd0 tx=0x7fe0c000c490 comp rx=0 tx=0).stop 2024-04-28T12:43:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.410+0000 7fe0cf200640 1 -- 172.21.15.131:0/2816238805 shutdown_connections 2024-04-28T12:43:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.410+0000 7fe0cf200640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe0a406c8f0 0x7fe0a406edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.410+0000 7fe0cf200640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c81467c0 0x7fe0c815c420 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.410+0000 7fe0cf200640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0c80d1060 0x7fe0c813ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.410+0000 7fe0cf200640 1 --2- 172.21.15.131:0/2816238805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0c8072b70 0x7fe0c815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.410+0000 7fe0cf200640 1 -- 172.21.15.131:0/2816238805 >> 172.21.15.131:0/2816238805 conn(0x7fe0c80b2900 msgr2=0x7fe0c80b4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:17.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.410+0000 7fe0cf200640 1 -- 172.21.15.131:0/2816238805 shutdown_connections 2024-04-28T12:43:17.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.410+0000 7fe0cf200640 1 -- 172.21.15.131:0/2816238805 wait complete. 2024-04-28T12:43:17.425 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-04-28T12:43:17.058631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237800,"kb_used_data":4992,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736920,"statfs":{"total":773094113280,"available":772850606080,"internally_reserved":0,"allocated":5111808,"data_stored":2900264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002152"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-28T12:43:15.025727+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:43:15.025727+0000","last_peered":"2024-04-28T12:43:15.025727+0000","last_clean":"2024-04-28T12:43:15.025727+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:43:15.025727+0000","last_undegraded":"2024-04-28T12:43:15.025727+0000","last_fullsized":"2024-04-28T12:43:15.025727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1085440,"data_stored":806027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27504,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":495616,"data_stored":216305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1073152,"data_stored":799637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:43:17.425 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:43:17.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.617+0000 7f91f3200640 1 Processor -- start 2024-04-28T12:43:17.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.617+0000 7f91f3200640 1 -- start start 2024-04-28T12:43:17.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.618+0000 7f91f3200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f91ec1560f0 0x7f91ec0d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.619+0000 7f91f3200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f91ec15c970 0x7f91ec0b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.619+0000 7f91f3200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91ec15cfd0 con 0x7f91ec0d1d20 2024-04-28T12:43:17.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.619+0000 7f91f3200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f91ec0d21d0 con 0x7f91ec1560f0 2024-04-28T12:43:17.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.619+0000 7f91f3200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f91ec15cda0 con 0x7f91ec15c970 2024-04-28T12:43:17.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.619+0000 7f91f1e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f91ec1560f0 0x7f91ec0d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.619+0000 7f91f1400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f91ec0d1d20 0x7f91ec15c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33160/0 (socket says 172.21.15.131:33160) 2024-04-28T12:43:17.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.619+0000 7f91f1400640 1 -- 172.21.15.131:0/3558655253 learned_addr learned my addr 172.21.15.131:0/3558655253 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:17.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.619+0000 7f91f2800640 1 --2- 172.21.15.131:0/3558655253 >> v2:172.21.15.157:3300/0 conn(0x7f91ec15c970 0x7f91ec0b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.619+0000 7f91f1e00640 1 -- 172.21.15.131:0/3558655253 >> v2:172.21.15.157:3300/0 conn(0x7f91ec15c970 msgr2=0x7f91ec0b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:17.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.619+0000 7f91f1e00640 1 --2- 172.21.15.131:0/3558655253 >> v2:172.21.15.157:3300/0 conn(0x7f91ec15c970 0x7f91ec0b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.619+0000 7f91f1e00640 1 -- 172.21.15.131:0/3558655253 >> v1:172.21.15.131:6789/0 conn(0x7f91ec0d1d20 legacy=0x7f91ec15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.619+0000 7f91f1e00640 1 -- 172.21.15.131:0/3558655253 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec1564d0 con 0x7f91ec1560f0 2024-04-28T12:43:17.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.620+0000 7f91f1e00640 1 --2- 172.21.15.131:0/3558655253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec1560f0 0x7f91ec0d17e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f91ec15d1a0 tx=0x7f91e002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58d45ddc59a9a2f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:17.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.621+0000 7f91f0a00640 1 -- 172.21.15.131:0/3558655253 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0037070 con 0x7f91ec1560f0 2024-04-28T12:43:17.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.621+0000 7f91f0a00640 1 -- 172.21.15.131:0/3558655253 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e002ae50 con 0x7f91ec1560f0 2024-04-28T12:43:17.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.621+0000 7f91f0a00640 1 -- 172.21.15.131:0/3558655253 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0032660 con 0x7f91ec1560f0 2024-04-28T12:43:17.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.621+0000 7f91f3200640 1 -- 172.21.15.131:0/3558655253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec1560f0 msgr2=0x7f91ec0d17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.621+0000 7f91f3200640 1 --2- 172.21.15.131:0/3558655253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec1560f0 0x7f91ec0d17e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f91ec15d1a0 tx=0x7f91e002a100 comp rx=0 tx=0).stop 2024-04-28T12:43:17.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.622+0000 7f91f3200640 1 -- 172.21.15.131:0/3558655253 shutdown_connections 2024-04-28T12:43:17.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.622+0000 7f91f3200640 1 --2- 172.21.15.131:0/3558655253 >> v2:172.21.15.157:3300/0 conn(0x7f91ec15c970 0x7f91ec0b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.622+0000 7f91f3200640 1 --2- 172.21.15.131:0/3558655253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec1560f0 0x7f91ec0d17e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.622+0000 7f91f3200640 1 -- 172.21.15.131:0/3558655253 >> 172.21.15.131:0/3558655253 conn(0x7f91ec075df0 msgr2=0x7f91ec068090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:17.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.622+0000 7f91f3200640 1 -- 172.21.15.131:0/3558655253 shutdown_connections 2024-04-28T12:43:17.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.622+0000 7f91f3200640 1 -- 172.21.15.131:0/3558655253 wait complete. 2024-04-28T12:43:17.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.623+0000 7f91f3200640 1 Processor -- start 2024-04-28T12:43:17.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.623+0000 7f91f3200640 1 -- start start 2024-04-28T12:43:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.624+0000 7f91f3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec0d1d20 0x7f91ec0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.624+0000 7f91f3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91ec1560f0 0x7f91ec14a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.624+0000 7f91f3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91ec15c970 0x7f91ec0c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.624+0000 7f91f3200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec15fd10 con 0x7f91ec0d1d20 2024-04-28T12:43:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.624+0000 7f91f3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec15fa30 con 0x7f91ec15c970 2024-04-28T12:43:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.624+0000 7f91f3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f91ec15fff0 con 0x7f91ec1560f0 2024-04-28T12:43:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.625+0000 7f91f1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec0d1d20 0x7f91ec0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.625+0000 7f91f1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec0d1d20 0x7f91ec0c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33830/0 (socket says 172.21.15.131:33830) 2024-04-28T12:43:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.625+0000 7f91f1e00640 1 -- 172.21.15.131:0/1626281893 learned_addr learned my addr 172.21.15.131:0/1626281893 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.625+0000 7f91f1e00640 1 -- 172.21.15.131:0/1626281893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91ec1560f0 msgr2=0x7f91ec14a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.625+0000 7f91f1e00640 1 --2- 172.21.15.131:0/1626281893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91ec1560f0 0x7f91ec14a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.625+0000 7f91f1e00640 1 -- 172.21.15.131:0/1626281893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91ec15c970 msgr2=0x7f91ec0c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.625+0000 7f91f1e00640 1 --2- 172.21.15.131:0/1626281893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91ec15c970 0x7f91ec0c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.625+0000 7f91f1e00640 1 -- 172.21.15.131:0/1626281893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec14ae80 con 0x7f91ec0d1d20 2024-04-28T12:43:17.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.625+0000 7f91f1e00640 1 --2- 172.21.15.131:0/1626281893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec0d1d20 0x7f91ec0c6770 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f91e0000c00 tx=0x7f91e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:17.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.626+0000 7f91e6000640 1 -- 172.21.15.131:0/1626281893 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0041070 con 0x7f91ec0d1d20 2024-04-28T12:43:17.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.626+0000 7f91e6000640 1 -- 172.21.15.131:0/1626281893 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e003c670 con 0x7f91ec0d1d20 2024-04-28T12:43:17.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.626+0000 7f91e6000640 1 -- 172.21.15.131:0/1626281893 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0037040 con 0x7f91ec0d1d20 2024-04-28T12:43:17.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.626+0000 7f91f3200640 1 -- 172.21.15.131:0/1626281893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ec0c6de0 con 0x7f91ec0d1d20 2024-04-28T12:43:17.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.626+0000 7f91f3200640 1 -- 172.21.15.131:0/1626281893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ec0c73a0 con 0x7f91ec0d1d20 2024-04-28T12:43:17.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.627+0000 7f91f3200640 1 -- 172.21.15.131:0/1626281893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ec069a60 con 0x7f91ec0d1d20 2024-04-28T12:43:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.628+0000 7f91e6000640 1 -- 172.21.15.131:0/1626281893 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91e0044070 con 0x7f91ec0d1d20 2024-04-28T12:43:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.629+0000 7f91e6000640 1 --2- 172.21.15.131:0/1626281893 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f91d006c920 0x7f91d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.629+0000 7f91e6000640 1 -- 172.21.15.131:0/1626281893 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f91e00ac0e0 con 0x7f91ec0d1d20 2024-04-28T12:43:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.629+0000 7f91f1400640 1 --2- 172.21.15.131:0/1626281893 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f91d006c920 0x7f91d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.631+0000 7f91f1400640 1 --2- 172.21.15.131:0/1626281893 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f91d006c920 0x7f91d006ede0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f91dc000f60 tx=0x7f91dc007a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:17.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.633+0000 7f91e6000640 1 -- 172.21.15.131:0/1626281893 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f91e007a670 con 0x7f91ec0d1d20 2024-04-28T12:43:17.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.830+0000 7f91f3200640 1 -- 172.21.15.131:0/1626281893 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f91ec06f0d0 con 0x7f91d006c920 2024-04-28T12:43:17.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.833+0000 7f91e6000640 1 -- 172.21.15.131:0/1626281893 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13489 (secure 0 0 0) 0x7f91ec06f0d0 con 0x7f91d006c920 2024-04-28T12:43:17.829 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:17.829 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:43:17.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.834+0000 7f91f3200640 1 -- 172.21.15.131:0/1626281893 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f91d006c920 msgr2=0x7f91d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.834+0000 7f91f3200640 1 --2- 172.21.15.131:0/1626281893 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f91d006c920 0x7f91d006ede0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f91dc000f60 tx=0x7f91dc007a80 comp rx=0 tx=0).stop 2024-04-28T12:43:17.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.835+0000 7f91f3200640 1 -- 172.21.15.131:0/1626281893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec0d1d20 msgr2=0x7f91ec0c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.835+0000 7f91f3200640 1 --2- 172.21.15.131:0/1626281893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec0d1d20 0x7f91ec0c6770 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f91e0000c00 tx=0x7f91e00046c0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.835+0000 7f91f3200640 1 -- 172.21.15.131:0/1626281893 shutdown_connections 2024-04-28T12:43:17.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.835+0000 7f91f3200640 1 --2- 172.21.15.131:0/1626281893 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f91d006c920 0x7f91d006ede0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.835+0000 7f91f3200640 1 --2- 172.21.15.131:0/1626281893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91ec15c970 0x7f91ec0c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.835+0000 7f91f3200640 1 --2- 172.21.15.131:0/1626281893 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91ec1560f0 0x7f91ec14a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.835+0000 7f91f3200640 1 --2- 172.21.15.131:0/1626281893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91ec0d1d20 0x7f91ec0c6770 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.835+0000 7f91f3200640 1 -- 172.21.15.131:0/1626281893 >> 172.21.15.131:0/1626281893 conn(0x7f91ec075df0 msgr2=0x7f91ec0b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:17.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.835+0000 7f91f3200640 1 -- 172.21.15.131:0/1626281893 shutdown_connections 2024-04-28T12:43:17.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:17.835+0000 7f91f3200640 1 -- 172.21.15.131:0/1626281893 wait complete. 2024-04-28T12:43:17.849 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-04-28T12:43:17.058631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237800,"kb_used_data":4992,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736920,"statfs":{"total":773094113280,"available":772850606080,"internally_reserved":0,"allocated":5111808,"data_stored":2900264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002152"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-28T12:43:15.025727+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:43:15.025727+0000","last_peered":"2024-04-28T12:43:15.025727+0000","last_clean":"2024-04-28T12:43:15.025727+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:43:15.025727+0000","last_undegraded":"2024-04-28T12:43:15.025727+0000","last_fullsized":"2024-04-28T12:43:15.025727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1085440,"data_stored":806027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27504,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":495616,"data_stored":216305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1073152,"data_stored":799637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:43:17.850 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-28T12:43:17.850 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:43:18.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.053+0000 7fbf73000640 1 Processor -- start 2024-04-28T12:43:18.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.053+0000 7fbf73000640 1 -- start start 2024-04-28T12:43:18.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.055+0000 7fbf73000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbf6c1467c0 0x7fbf6c072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.055+0000 7fbf73000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6c146ba0 con 0x7fbf6c0b3ce0 2024-04-28T12:43:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.055+0000 7fbf73000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6c0731c0 con 0x7fbf6c0b4140 2024-04-28T12:43:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.055+0000 7fbf73000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fbf6c073dd0 con 0x7fbf6c1467c0 2024-04-28T12:43:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.055+0000 7fbf6be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbf6c1467c0 0x7fbf6c072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.055+0000 7fbf6b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbf6c0b3ce0 0x7fbf6c0d0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33166/0 (socket says 172.21.15.131:33166) 2024-04-28T12:43:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.055+0000 7fbf6b400640 1 -- 172.21.15.131:0/739422663 learned_addr learned my addr 172.21.15.131:0/739422663 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:18.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.056+0000 7fbf6be00640 1 -- 172.21.15.131:0/739422663 >> v1:172.21.15.157:6789/0 conn(0x7fbf6c0b4140 legacy=0x7fbf6c06dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:18.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.056+0000 7fbf6be00640 1 -- 172.21.15.131:0/739422663 >> v1:172.21.15.131:6789/0 conn(0x7fbf6c0b3ce0 legacy=0x7fbf6c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.056+0000 7fbf6be00640 1 -- 172.21.15.131:0/739422663 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c0734a0 con 0x7fbf6c1467c0 2024-04-28T12:43:18.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.057+0000 7fbf6be00640 1 --2- 172.21.15.131:0/739422663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6c1467c0 0x7fbf6c072b70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0097a0 tx=0x7fbf5c02a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cbe89a1cf8b07b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:18.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.058+0000 7fbf6aa00640 1 -- 172.21.15.131:0/739422663 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c037070 con 0x7fbf6c1467c0 2024-04-28T12:43:18.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.058+0000 7fbf6aa00640 1 -- 172.21.15.131:0/739422663 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5c02ad60 con 0x7fbf6c1467c0 2024-04-28T12:43:18.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.058+0000 7fbf6aa00640 1 -- 172.21.15.131:0/739422663 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c032660 con 0x7fbf6c1467c0 2024-04-28T12:43:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.058+0000 7fbf73000640 1 -- 172.21.15.131:0/739422663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6c1467c0 msgr2=0x7fbf6c072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.058+0000 7fbf73000640 1 --2- 172.21.15.131:0/739422663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6c1467c0 0x7fbf6c072b70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0097a0 tx=0x7fbf5c02a0d0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.058+0000 7fbf73000640 1 -- 172.21.15.131:0/739422663 shutdown_connections 2024-04-28T12:43:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.058+0000 7fbf73000640 1 --2- 172.21.15.131:0/739422663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6c1467c0 0x7fbf6c072b70 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.058+0000 7fbf73000640 1 -- 172.21.15.131:0/739422663 >> 172.21.15.131:0/739422663 conn(0x7fbf6c0b2900 msgr2=0x7fbf6c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.059+0000 7fbf73000640 1 -- 172.21.15.131:0/739422663 shutdown_connections 2024-04-28T12:43:18.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.059+0000 7fbf73000640 1 -- 172.21.15.131:0/739422663 wait complete. 2024-04-28T12:43:18.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.061+0000 7fbf73000640 1 Processor -- start 2024-04-28T12:43:18.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.061+0000 7fbf73000640 1 -- start start 2024-04-28T12:43:18.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.062+0000 7fbf73000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf6c0b3ce0 0x7fbf6c1f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.062+0000 7fbf73000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6c1467c0 0x7fbf6c0c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.062+0000 7fbf73000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf6c0c8620 0x7fbf6c15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.062+0000 7fbf73000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c0d1230 con 0x7fbf6c1467c0 2024-04-28T12:43:18.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.062+0000 7fbf73000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c073a60 con 0x7fbf6c0c8620 2024-04-28T12:43:18.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.062+0000 7fbf73000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c0d13b0 con 0x7fbf6c0b3ce0 2024-04-28T12:43:18.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.062+0000 7fbf6b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6c1467c0 0x7fbf6c0c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.062+0000 7fbf6b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6c1467c0 0x7fbf6c0c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33846/0 (socket says 172.21.15.131:33846) 2024-04-28T12:43:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.062+0000 7fbf6b400640 1 -- 172.21.15.131:0/1680670541 learned_addr learned my addr 172.21.15.131:0/1680670541 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.063+0000 7fbf6be00640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf6c0b3ce0 0x7fbf6c1f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.063+0000 7fbf6b400640 1 -- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf6c0b3ce0 msgr2=0x7fbf6c1f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.063+0000 7fbf71000640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf6c0c8620 0x7fbf6c15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.063+0000 7fbf6b400640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf6c0b3ce0 0x7fbf6c1f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.063+0000 7fbf6b400640 1 -- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf6c0c8620 msgr2=0x7fbf6c15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.063+0000 7fbf6b400640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf6c0c8620 0x7fbf6c15df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.063+0000 7fbf6b400640 1 -- 172.21.15.131:0/1680670541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c15ee20 con 0x7fbf6c1467c0 2024-04-28T12:43:18.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.063+0000 7fbf6b400640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6c1467c0 0x7fbf6c0c80e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c780 tx=0x7fbf5800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.063+0000 7fbf63e00640 1 -- 172.21.15.131:0/1680670541 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58018070 con 0x7fbf6c1467c0 2024-04-28T12:43:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.063+0000 7fbf63e00640 1 -- 172.21.15.131:0/1680670541 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf580072f0 con 0x7fbf6c1467c0 2024-04-28T12:43:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.063+0000 7fbf63e00640 1 -- 172.21.15.131:0/1680670541 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf580047e0 con 0x7fbf6c1467c0 2024-04-28T12:43:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.063+0000 7fbf73000640 1 -- 172.21.15.131:0/1680670541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c0c5c10 con 0x7fbf6c1467c0 2024-04-28T12:43:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.064+0000 7fbf73000640 1 -- 172.21.15.131:0/1680670541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c0c61d0 con 0x7fbf6c1467c0 2024-04-28T12:43:18.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.065+0000 7fbf73000640 1 -- 172.21.15.131:0/1680670541 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c0b57b0 con 0x7fbf6c1467c0 2024-04-28T12:43:18.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.066+0000 7fbf63e00640 1 -- 172.21.15.131:0/1680670541 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf58004980 con 0x7fbf6c1467c0 2024-04-28T12:43:18.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.067+0000 7fbf63e00640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbf4c06c920 0x7fbf4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.067+0000 7fbf63e00640 1 -- 172.21.15.131:0/1680670541 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fbf5808ae70 con 0x7fbf6c1467c0 2024-04-28T12:43:18.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.068+0000 7fbf6be00640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbf4c06c920 0x7fbf4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.070+0000 7fbf6be00640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbf4c06c920 0x7fbf4c06ede0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c034000 tx=0x7fbf5c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:18.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.072+0000 7fbf63e00640 1 -- 172.21.15.131:0/1680670541 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf6c0b57b0 con 0x7fbf6c1467c0 2024-04-28T12:43:18.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.270+0000 7fbf73000640 1 -- 172.21.15.131:0/1680670541 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf6c0b0360 con 0x7fbf4c06c920 2024-04-28T12:43:18.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.273+0000 7fbf63e00640 1 -- 172.21.15.131:0/1680670541 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13489 (secure 0 0 0) 0x7fbf6c0b0360 con 0x7fbf4c06c920 2024-04-28T12:43:18.269 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:18.269 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:43:18.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.275+0000 7fbf73000640 1 -- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbf4c06c920 msgr2=0x7fbf4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.275+0000 7fbf73000640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbf4c06c920 0x7fbf4c06ede0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c034000 tx=0x7fbf5c02e000 comp rx=0 tx=0).stop 2024-04-28T12:43:18.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.275+0000 7fbf73000640 1 -- 172.21.15.131:0/1680670541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6c1467c0 msgr2=0x7fbf6c0c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.275+0000 7fbf73000640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6c1467c0 0x7fbf6c0c80e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c780 tx=0x7fbf5800cc40 comp rx=0 tx=0).stop 2024-04-28T12:43:18.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.275+0000 7fbf73000640 1 -- 172.21.15.131:0/1680670541 shutdown_connections 2024-04-28T12:43:18.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.275+0000 7fbf73000640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbf4c06c920 0x7fbf4c06ede0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.275+0000 7fbf73000640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf6c0c8620 0x7fbf6c15df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.275+0000 7fbf73000640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6c1467c0 0x7fbf6c0c80e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.275+0000 7fbf73000640 1 --2- 172.21.15.131:0/1680670541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf6c0b3ce0 0x7fbf6c1f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.275+0000 7fbf73000640 1 -- 172.21.15.131:0/1680670541 >> 172.21.15.131:0/1680670541 conn(0x7fbf6c0b2900 msgr2=0x7fbf6c145260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:18.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.275+0000 7fbf73000640 1 -- 172.21.15.131:0/1680670541 shutdown_connections 2024-04-28T12:43:18.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.275+0000 7fbf73000640 1 -- 172.21.15.131:0/1680670541 wait complete. 2024-04-28T12:43:18.289 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-04-28T12:43:17.058631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237800,"kb_used_data":4992,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736920,"statfs":{"total":773094113280,"available":772850606080,"internally_reserved":0,"allocated":5111808,"data_stored":2900264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002152"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-28T12:43:15.025727+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:43:15.025727+0000","last_peered":"2024-04-28T12:43:15.025727+0000","last_clean":"2024-04-28T12:43:15.025727+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:43:15.025727+0000","last_undegraded":"2024-04-28T12:43:15.025727+0000","last_fullsized":"2024-04-28T12:43:15.025727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1085440,"data_stored":806027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27504,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":495616,"data_stored":216305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":215659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1073152,"data_stored":799637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:43:18.290 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:43:18.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.484+0000 7f228a800640 1 Processor -- start 2024-04-28T12:43:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.484+0000 7f228a800640 1 -- start start 2024-04-28T12:43:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.486+0000 7f228a800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f228415f0d0 0x7f22840c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.486+0000 7f228a800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22840c8830 con 0x7f22840b71a0 2024-04-28T12:43:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.486+0000 7f228a800640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22840d3cb0 con 0x7f22840b7580 2024-04-28T12:43:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.486+0000 7f228a800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f22840c8af0 con 0x7f228415f0d0 2024-04-28T12:43:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.486+0000 7f2283400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f22840b71a0 0x7f22840d34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33180/0 (socket says 172.21.15.131:33180) 2024-04-28T12:43:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.486+0000 7f2283400640 1 -- 172.21.15.131:0/525025622 learned_addr learned my addr 172.21.15.131:0/525025622 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:18.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.486+0000 7f2283e00640 1 --2- 172.21.15.131:0/525025622 >> v2:172.21.15.157:3300/0 conn(0x7f228415f0d0 0x7f22840c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3376207425 0 0) 0x7f22840c8830 con 0x7f22840b71a0 2024-04-28T12:43:18.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2268003520 con 0x7f22840b71a0 2024-04-28T12:43:18.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3267509609 0 0) 0x7f2268003520 con 0x7f22840b71a0 2024-04-28T12:43:18.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 >> v2:172.21.15.157:3300/0 conn(0x7f228415f0d0 msgr2=0x7f22840c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 --2- 172.21.15.131:0/525025622 >> v2:172.21.15.157:3300/0 conn(0x7f228415f0d0 0x7f22840c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 >> v1:172.21.15.157:6789/0 conn(0x7f22840b7580 legacy=0x7f228415e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22840c8dd0 con 0x7f22840b71a0 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f22740019f0 con 0x7f22840b71a0 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 >> v1:172.21.15.131:6789/0 conn(0x7f22840b71a0 legacy=0x7f22840d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 --2- 172.21.15.131:0/525025622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2268003a00 0x7f2268003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 --2- 172.21.15.131:0/525025622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2268004800 0x7f2268004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 --2- 172.21.15.131:0/525025622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2268005210 0x7f226800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f22840c8af0 con 0x7f2268004800 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f226800c160 con 0x7f2268003a00 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.487+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f226800c360 con 0x7f2268005210 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.488+0000 7f227aa00640 1 --2- 172.21.15.131:0/525025622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2268004800 0x7f2268004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.488+0000 7f2283400640 1 --2- 172.21.15.131:0/525025622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2268003a00 0x7f2268003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.488+0000 7f2283e00640 1 --2- 172.21.15.131:0/525025622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2268005210 0x7f226800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.488+0000 7f227aa00640 1 -- 172.21.15.131:0/525025622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2268005210 msgr2=0x7f226800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:18.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.488+0000 7f227aa00640 1 --2- 172.21.15.131:0/525025622 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2268005210 0x7f226800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.488+0000 7f227aa00640 1 -- 172.21.15.131:0/525025622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2268003a00 msgr2=0x7f2268003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:18.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.488+0000 7f227aa00640 1 --2- 172.21.15.131:0/525025622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2268003a00 0x7f2268003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.488+0000 7f227aa00640 1 -- 172.21.15.131:0/525025622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2268003520 con 0x7f2268004800 2024-04-28T12:43:18.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.488+0000 7f2283400640 1 --2- 172.21.15.131:0/525025622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2268003a00 0x7f2268003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:18.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.488+0000 7f2283e00640 1 -- 172.21.15.131:0/525025622 reap_dead start 2024-04-28T12:43:18.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.489+0000 7f227aa00640 1 --2- 172.21.15.131:0/525025622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2268004800 0x7f2268004c80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f22840b7960 tx=0x7f2268005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdf9fa35d562430 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:18.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.489+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c001370 con 0x7f2268004800 2024-04-28T12:43:18.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.489+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c0052d0 con 0x7f2268004800 2024-04-28T12:43:18.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.489+0000 7f2282a00640 1 -- 172.21.15.131:0/525025622 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c02ab40 con 0x7f2268004800 2024-04-28T12:43:18.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.490+0000 7f228a800640 1 -- 172.21.15.131:0/525025622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2268004800 msgr2=0x7f2268004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.490+0000 7f228a800640 1 --2- 172.21.15.131:0/525025622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2268004800 0x7f2268004c80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f22840b7960 tx=0x7f2268005830 comp rx=0 tx=0).stop 2024-04-28T12:43:18.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.490+0000 7f228a800640 1 -- 172.21.15.131:0/525025622 shutdown_connections 2024-04-28T12:43:18.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.490+0000 7f228a800640 1 --2- 172.21.15.131:0/525025622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2268004800 0x7f2268004c80 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.490+0000 7f228a800640 1 -- 172.21.15.131:0/525025622 >> 172.21.15.131:0/525025622 conn(0x7f22840d4d70 msgr2=0x7f22840d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:18.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.490+0000 7f228a800640 1 -- 172.21.15.131:0/525025622 shutdown_connections 2024-04-28T12:43:18.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.490+0000 7f228a800640 1 -- 172.21.15.131:0/525025622 wait complete. 2024-04-28T12:43:18.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.492+0000 7f228a800640 1 Processor -- start 2024-04-28T12:43:18.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.492+0000 7f228a800640 1 -- start start 2024-04-28T12:43:18.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.493+0000 7f228a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2268004800 0x7f22841491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.493+0000 7f228a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f228415f820 0x7f22840b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.493+0000 7f228a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228415fca0 0x7f22840ba070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.493+0000 7f228a800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f228415f6a0 con 0x7f228415fca0 2024-04-28T12:43:18.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.493+0000 7f228a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f228415f520 con 0x7f228415f820 2024-04-28T12:43:18.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.493+0000 7f228a800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f22840d34d0 con 0x7f2268004800 2024-04-28T12:43:18.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.494+0000 7f2283e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228415fca0 0x7f22840ba070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.494+0000 7f2283e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228415fca0 0x7f22840ba070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33868/0 (socket says 172.21.15.131:33868) 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.494+0000 7f2283e00640 1 -- 172.21.15.131:0/4161561096 learned_addr learned my addr 172.21.15.131:0/4161561096 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.494+0000 7f2283400640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2268004800 0x7f22841491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.494+0000 7f227aa00640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f228415f820 0x7f22840b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.494+0000 7f2283e00640 1 -- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2268004800 msgr2=0x7f22841491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.494+0000 7f2283e00640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2268004800 0x7f22841491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.494+0000 7f2283e00640 1 -- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f228415f820 msgr2=0x7f22840b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.494+0000 7f2283e00640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f228415f820 0x7f22840b9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.494+0000 7f2283e00640 1 -- 172.21.15.131:0/4161561096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228414caa0 con 0x7f228415fca0 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.494+0000 7f227aa00640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f228415f820 0x7f22840b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.494+0000 7f2283e00640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228415fca0 0x7f22840ba070 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f22840bae00 tx=0x7f227c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.495+0000 7f227be00640 1 -- 172.21.15.131:0/4161561096 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c00f040 con 0x7f228415fca0 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.495+0000 7f227be00640 1 -- 172.21.15.131:0/4161561096 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227c01c070 con 0x7f228415fca0 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.495+0000 7f227be00640 1 -- 172.21.15.131:0/4161561096 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c00ead0 con 0x7f228415fca0 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.495+0000 7f228a800640 1 -- 172.21.15.131:0/4161561096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22841f8820 con 0x7f228415fca0 2024-04-28T12:43:18.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.495+0000 7f228a800640 1 -- 172.21.15.131:0/4161561096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22841f8de0 con 0x7f228415fca0 2024-04-28T12:43:18.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.496+0000 7f228a800640 1 -- 172.21.15.131:0/4161561096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2284069350 con 0x7f228415fca0 2024-04-28T12:43:18.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.497+0000 7f227be00640 1 -- 172.21.15.131:0/4161561096 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f227c00ec70 con 0x7f228415fca0 2024-04-28T12:43:18.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.498+0000 7f227be00640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f226006c8f0 0x7f226006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:18.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.498+0000 7f227be00640 1 -- 172.21.15.131:0/4161561096 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f227c08b990 con 0x7f228415fca0 2024-04-28T12:43:18.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.499+0000 7f2283400640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f226006c8f0 0x7f226006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:18.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.500+0000 7f2283400640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f226006c8f0 0x7f226006edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2274001b60 tx=0x7f2274009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:18.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.502+0000 7f227be00640 1 -- 172.21.15.131:0/4161561096 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f228414caa0 con 0x7f228415fca0 2024-04-28T12:43:18.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.704+0000 7f228a800640 1 -- 172.21.15.131:0/4161561096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f228408ee00 con 0x7f228415fca0 2024-04-28T12:43:18.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.706+0000 7f227be00640 1 -- 172.21.15.131:0/4161561096 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+13317 (secure 0 0 0) 0x7f227c0598e0 con 0x7f228415fca0 2024-04-28T12:43:18.703 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:18.703 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":43,"fsid":"fc0efa02-d382-4f6a-947e-cbe7ab2d58d1","created":"2024-04-28T12:37:01.816886+0000","modified":"2024-04-28T12:43:14.955345+0000","last_up_change":"2024-04-28T12:37:14.661042+0000","last_in_change":"2024-04-28T12:37:06.178042+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:37:14.943463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"15970349-8188-4409-af6d-1ac6680cddd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6813","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6816","nonce":1049412824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6819","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6821","nonce":1049412824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6830","nonce":1049412824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1049412824},{"type":"v1","addr":"172.21.15.131:6825","nonce":1049412824}]},"public_addr":"172.21.15.131:6816/1049412824","cluster_addr":"172.21.15.131:6821/1049412824","heartbeat_back_addr":"172.21.15.131:6830/1049412824","heartbeat_front_addr":"172.21.15.131:6825/1049412824","state":["exists","up"]},{"osd":1,"uuid":"9ece051d-6eca-401a-b213-c8a72199d9d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6801","nonce":703178662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6803","nonce":703178662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6807","nonce":703178662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":703178662},{"type":"v1","addr":"172.21.15.131:6805","nonce":703178662}]},"public_addr":"172.21.15.131:6801/703178662","cluster_addr":"172.21.15.131:6803/703178662","heartbeat_back_addr":"172.21.15.131:6807/703178662","heartbeat_front_addr":"172.21.15.131:6805/703178662","state":["exists","up"]},{"osd":2,"uuid":"e1f72f87-0adf-410a-aa9c-d03196484282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6809","nonce":768553345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6811","nonce":768553345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6826","nonce":768553345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":768553345},{"type":"v1","addr":"172.21.15.131:6818","nonce":768553345}]},"public_addr":"172.21.15.131:6809/768553345","cluster_addr":"172.21.15.131:6811/768553345","heartbeat_back_addr":"172.21.15.131:6826/768553345","heartbeat_front_addr":"172.21.15.131:6818/768553345","state":["exists","up"]},{"osd":3,"uuid":"7097fba1-6608-41a1-833b-582c898e3cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6815","nonce":708415318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6817","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6820","nonce":708415318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6829","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6831","nonce":708415318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6823","nonce":708415318},{"type":"v1","addr":"172.21.15.131:6827","nonce":708415318}]},"public_addr":"172.21.15.131:6815/708415318","cluster_addr":"172.21.15.131:6820/708415318","heartbeat_back_addr":"172.21.15.131:6831/708415318","heartbeat_front_addr":"172.21.15.131:6827/708415318","state":["exists","up"]},{"osd":4,"uuid":"f6717c55-ded0-4f5c-8cfb-e8f8d0b763b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6803","nonce":4053377461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6805","nonce":4053377461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6809","nonce":4053377461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":4053377461},{"type":"v1","addr":"172.21.15.157:6807","nonce":4053377461}]},"public_addr":"172.21.15.157:6803/4053377461","cluster_addr":"172.21.15.157:6805/4053377461","heartbeat_back_addr":"172.21.15.157:6809/4053377461","heartbeat_front_addr":"172.21.15.157:6807/4053377461","state":["exists","up"]},{"osd":5,"uuid":"7549d7c2-0964-4a6f-abe9-a0ba585e1267","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6819","nonce":2721118630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6822","nonce":2721118630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6830","nonce":2721118630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2721118630},{"type":"v1","addr":"172.21.15.157:6826","nonce":2721118630}]},"public_addr":"172.21.15.157:6819/2721118630","cluster_addr":"172.21.15.157:6822/2721118630","heartbeat_back_addr":"172.21.15.157:6830/2721118630","heartbeat_front_addr":"172.21.15.157:6826/2721118630","state":["exists","up"]},{"osd":6,"uuid":"7570ea49-d73f-4998-a6b5-e0cfa5162fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6811","nonce":2594435113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6813","nonce":2594435113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6817","nonce":2594435113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2594435113},{"type":"v1","addr":"172.21.15.157:6815","nonce":2594435113}]},"public_addr":"172.21.15.157:6811/2594435113","cluster_addr":"172.21.15.157:6813/2594435113","heartbeat_back_addr":"172.21.15.157:6817/2594435113","heartbeat_front_addr":"172.21.15.157:6815/2594435113","state":["exists","up"]},{"osd":7,"uuid":"6c98a122-d9fc-43fa-8b4e-9934e54e369d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6821","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6823","nonce":2246043568}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6825","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6827","nonce":2246043568}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6833","nonce":2246043568}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6829","nonce":2246043568},{"type":"v1","addr":"172.21.15.157:6831","nonce":2246043568}]},"public_addr":"172.21.15.157:6823/2246043568","cluster_addr":"172.21.15.157:6827/2246043568","heartbeat_back_addr":"172.21.15.157:6833/2246043568","heartbeat_front_addr":"172.21.15.157:6831/2246043568","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.571054+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.304395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.608970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.820293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.739103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.660669+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.351840+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:37:11.599973+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6837/837933884":"2024-04-29T12:43:11.935127+0000","172.21.15.157:6835/837933884":"2024-04-29T12:43:11.935127+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:43:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.709+0000 7f228a800640 1 -- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f226006c8f0 msgr2=0x7f226006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.709+0000 7f228a800640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f226006c8f0 0x7f226006edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2274001b60 tx=0x7f2274009040 comp rx=0 tx=0).stop 2024-04-28T12:43:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.709+0000 7f228a800640 1 -- 172.21.15.131:0/4161561096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228415fca0 msgr2=0x7f22840ba070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.709+0000 7f228a800640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228415fca0 0x7f22840ba070 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f22840bae00 tx=0x7f227c010c20 comp rx=0 tx=0).stop 2024-04-28T12:43:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.709+0000 7f228a800640 1 -- 172.21.15.131:0/4161561096 shutdown_connections 2024-04-28T12:43:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.709+0000 7f228a800640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f226006c8f0 0x7f226006edb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.709+0000 7f228a800640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228415fca0 0x7f22840ba070 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.709+0000 7f228a800640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f228415f820 0x7f22840b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.709+0000 7f228a800640 1 --2- 172.21.15.131:0/4161561096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2268004800 0x7f22841491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.709+0000 7f228a800640 1 -- 172.21.15.131:0/4161561096 >> 172.21.15.131:0/4161561096 conn(0x7f22840d4d70 msgr2=0x7f228406dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.710+0000 7f228a800640 1 -- 172.21.15.131:0/4161561096 shutdown_connections 2024-04-28T12:43:18.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:18.710+0000 7f228a800640 1 -- 172.21.15.131:0/4161561096 wait complete. 2024-04-28T12:43:19.725 INFO:tasks.ceph:Scrubbing osd.0 2024-04-28T12:43:19.726 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-28T12:43:19.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.930+0000 7fc52de00640 1 Processor -- start 2024-04-28T12:43:19.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.930+0000 7fc52de00640 1 -- start start 2024-04-28T12:43:19.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.931+0000 7fc52de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280cdc80 0x7fc5280ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:19.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.931+0000 7fc52de00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52815d260 con 0x7fc5280b3ab0 2024-04-28T12:43:19.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.931+0000 7fc52de00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52815d540 con 0x7fc5280b00a0 2024-04-28T12:43:19.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.931+0000 7fc52de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc5280ce600 con 0x7fc5280cdc80 2024-04-28T12:43:19.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.932+0000 7fc52d400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc5280b3ab0 0x7fc5280b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33192/0 (socket says 172.21.15.131:33192) 2024-04-28T12:43:19.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.932+0000 7fc52d400640 1 -- 172.21.15.131:0/1178599962 learned_addr learned my addr 172.21.15.131:0/1178599962 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:19.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.932+0000 7fc527e00640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280cdc80 0x7fc5280ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:19.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.932+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277233927 0 0) 0x7fc52815d260 con 0x7fc5280b3ab0 2024-04-28T12:43:19.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.932+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc510003520 con 0x7fc5280b3ab0 2024-04-28T12:43:19.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.932+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3976566687 0 0) 0x7fc510003520 con 0x7fc5280b3ab0 2024-04-28T12:43:19.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280cdc80 msgr2=0x7fc5280ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527400640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280cdc80 0x7fc5280ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 >> v1:172.21.15.157:6789/0 conn(0x7fc5280b00a0 legacy=0x7fc5280cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52815d820 con 0x7fc5280b3ab0 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fc5200019f0 con 0x7fc5280b3ab0 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 >> v1:172.21.15.131:6789/0 conn(0x7fc5280b3ab0 legacy=0x7fc5280b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527400640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280cdc80 0x7fc5100039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527400640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc510003fb0 0x7fc510004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527400640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510004d40 0x7fc5100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc5280ce600 con 0x7fc510003fb0 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc51000bb00 con 0x7fc510004d40 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc51000bd00 con 0x7fc5280cdc80 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527e00640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc510003fb0 0x7fc510004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc52ca00640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280cdc80 0x7fc5100039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527e00640 1 -- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280cdc80 msgr2=0x7fc5100039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc52d400640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510004d40 0x7fc5100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527e00640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280cdc80 0x7fc5100039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527e00640 1 -- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510004d40 msgr2=0x7fc5100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527e00640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510004d40 0x7fc5100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.933+0000 7fc527e00640 1 -- 172.21.15.131:0/1178599962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc510003520 con 0x7fc510003fb0 2024-04-28T12:43:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.934+0000 7fc52ca00640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280cdc80 0x7fc5100039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:19.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.934+0000 7fc52d400640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510004d40 0x7fc5100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:19.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.935+0000 7fc527e00640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc510003fb0 0x7fc510004800 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc514009ac0 tx=0x7fc510005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f865de58ee3da1a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:19.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.935+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514036070 con 0x7fc510003fb0 2024-04-28T12:43:19.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.935+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc514031400 con 0x7fc510003fb0 2024-04-28T12:43:19.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.935+0000 7fc527400640 1 -- 172.21.15.131:0/1178599962 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514009690 con 0x7fc510003fb0 2024-04-28T12:43:19.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.935+0000 7fc52de00640 1 -- 172.21.15.131:0/1178599962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc510003fb0 msgr2=0x7fc510004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:19.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.935+0000 7fc52de00640 1 --2- 172.21.15.131:0/1178599962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc510003fb0 0x7fc510004800 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc514009ac0 tx=0x7fc510005830 comp rx=0 tx=0).stop 2024-04-28T12:43:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.935+0000 7fc52d400640 1 -- 172.21.15.131:0/1178599962 reap_dead start 2024-04-28T12:43:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.935+0000 7fc52de00640 1 -- 172.21.15.131:0/1178599962 shutdown_connections 2024-04-28T12:43:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.935+0000 7fc52de00640 1 -- 172.21.15.131:0/1178599962 >> 172.21.15.131:0/1178599962 conn(0x7fc5280b26b0 msgr2=0x7fc5280b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.936+0000 7fc52de00640 1 -- 172.21.15.131:0/1178599962 shutdown_connections 2024-04-28T12:43:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.936+0000 7fc52de00640 1 -- 172.21.15.131:0/1178599962 wait complete. 2024-04-28T12:43:19.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.937+0000 7fc52de00640 1 Processor -- start 2024-04-28T12:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.937+0000 7fc52de00640 1 -- start start 2024-04-28T12:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.938+0000 7fc52de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510003fb0 0x7fc5281f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.938+0000 7fc52de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280ad1e0 0x7fc5280cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.938+0000 7fc52de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc528140400 0x7fc528144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.938+0000 7fc52de00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc52806f670 con 0x7fc528140400 2024-04-28T12:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.938+0000 7fc52de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5280d0580 con 0x7fc510003fb0 2024-04-28T12:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.938+0000 7fc52de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc52806f7f0 con 0x7fc5280ad1e0 2024-04-28T12:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.939+0000 7fc52d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc528140400 0x7fc528144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:19.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.939+0000 7fc52d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc528140400 0x7fc528144660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33894/0 (socket says 172.21.15.131:33894) 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.939+0000 7fc52d400640 1 -- 172.21.15.131:0/3253122645 learned_addr learned my addr 172.21.15.131:0/3253122645 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.939+0000 7fc527e00640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280ad1e0 0x7fc5280cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.939+0000 7fc52d400640 1 -- 172.21.15.131:0/3253122645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280ad1e0 msgr2=0x7fc5280cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.939+0000 7fc52ca00640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510003fb0 0x7fc5281f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.939+0000 7fc52d400640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280ad1e0 0x7fc5280cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.939+0000 7fc52d400640 1 -- 172.21.15.131:0/3253122645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510003fb0 msgr2=0x7fc5281f7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.939+0000 7fc52d400640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc510003fb0 0x7fc5281f7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.939+0000 7fc52d400640 1 -- 172.21.15.131:0/3253122645 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5280ad950 con 0x7fc528140400 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.939+0000 7fc527e00640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5280ad1e0 0x7fc5280cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.939+0000 7fc52d400640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc528140400 0x7fc528144660 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc5280cd660 tx=0x7fc52000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.940+0000 7fc524c00640 1 -- 172.21.15.131:0/3253122645 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52000de70 con 0x7fc528140400 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.940+0000 7fc524c00640 1 -- 172.21.15.131:0/3253122645 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc52000bb80 con 0x7fc528140400 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.940+0000 7fc52de00640 1 -- 172.21.15.131:0/3253122645 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5280adc30 con 0x7fc528140400 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.940+0000 7fc524c00640 1 -- 172.21.15.131:0/3253122645 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520007070 con 0x7fc528140400 2024-04-28T12:43:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.940+0000 7fc52de00640 1 -- 172.21.15.131:0/3253122645 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5280ae1f0 con 0x7fc528140400 2024-04-28T12:43:19.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.941+0000 7fc52de00640 1 -- 172.21.15.131:0/3253122645 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc52806fe80 con 0x7fc528140400 2024-04-28T12:43:19.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.942+0000 7fc524c00640 1 -- 172.21.15.131:0/3253122645 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc52001b020 con 0x7fc528140400 2024-04-28T12:43:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.943+0000 7fc524c00640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc50806c920 0x7fc50806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.943+0000 7fc524c00640 1 -- 172.21.15.131:0/3253122645 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fc52008bc10 con 0x7fc528140400 2024-04-28T12:43:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.943+0000 7fc524c00640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] conn(0x7fc508072dc0 0x7fc508075200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.943+0000 7fc524c00640 1 -- 172.21.15.131:0/3253122645 --> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5080758d0 con 0x7fc508072dc0 2024-04-28T12:43:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.943+0000 7fc524c00640 1 -- 172.21.15.131:0/3253122645 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc5200902a0 con 0x7fc528140400 2024-04-28T12:43:19.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.943+0000 7fc52ca00640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc50806c920 0x7fc50806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:19.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.943+0000 7fc527e00640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] conn(0x7fc508072dc0 0x7fc508075200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:19.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.944+0000 7fc527e00640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] conn(0x7fc508072dc0 0x7fc508075200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:19.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.944+0000 7fc52ca00640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc50806c920 0x7fc50806ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc51c001630 tx=0x7fc51c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:19.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.945+0000 7fc524c00640 1 -- 172.21.15.131:0/3253122645 <== osd.0 v2:172.21.15.131:6813/1049412824 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc5080758d0 con 0x7fc508072dc0 2024-04-28T12:43:19.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.959+0000 7fc52de00640 1 -- 172.21.15.131:0/3253122645 --> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc5280716e0 con 0x7fc508072dc0 2024-04-28T12:43:19.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.960+0000 7fc524c00640 1 -- 172.21.15.131:0/3253122645 <== osd.0 v2:172.21.15.131:6813/1049412824 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc5280716e0 con 0x7fc508072dc0 2024-04-28T12:43:19.956 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:43:19.956 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T12:43:19.956 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:43:19.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.960+0000 7fc52de00640 1 -- 172.21.15.131:0/3253122645 >> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] conn(0x7fc508072dc0 msgr2=0x7fc508075200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:19.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.960+0000 7fc52de00640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.131:6813/1049412824,v1:172.21.15.131:6816/1049412824] conn(0x7fc508072dc0 0x7fc508075200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:19.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.960+0000 7fc52de00640 1 -- 172.21.15.131:0/3253122645 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc50806c920 msgr2=0x7fc50806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:19.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.960+0000 7fc52de00640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc50806c920 0x7fc50806ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc51c001630 tx=0x7fc51c009040 comp rx=0 tx=0).stop 2024-04-28T12:43:19.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.960+0000 7fc52de00640 1 -- 172.21.15.131:0/3253122645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc528140400 msgr2=0x7fc528144660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:19.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.960+0000 7fc52de00640 1 --2- 172.21.15.131:0/3253122645 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc528140400 0x7fc528144660 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc5280cd660 tx=0x7fc52000b4a0 comp rx=0 tx=0).stop 2024-04-28T12:43:19.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.960+0000 7fc52d400640 1 -- 172.21.15.131:0/3253122645 reap_dead start 2024-04-28T12:43:19.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.961+0000 7fc52de00640 1 -- 172.21.15.131:0/3253122645 shutdown_connections 2024-04-28T12:43:19.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.961+0000 7fc52de00640 1 -- 172.21.15.131:0/3253122645 >> 172.21.15.131:0/3253122645 conn(0x7fc5280b26b0 msgr2=0x7fc5280cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:19.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.961+0000 7fc52de00640 1 -- 172.21.15.131:0/3253122645 shutdown_connections 2024-04-28T12:43:19.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:19.961+0000 7fc52de00640 1 -- 172.21.15.131:0/3253122645 wait complete. 2024-04-28T12:43:19.973 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-28T12:43:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.154+0000 7fc509600640 1 Processor -- start 2024-04-28T12:43:20.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.154+0000 7fc509600640 1 -- start start 2024-04-28T12:43:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.156+0000 7fc509600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc504145330 0x7fc5040b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.156+0000 7fc509600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc504145710 con 0x7fc5041628e0 2024-04-28T12:43:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.156+0000 7fc509600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc504073dc0 con 0x7fc5040afe30 2024-04-28T12:43:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.156+0000 7fc509600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc5040b0250 con 0x7fc504145330 2024-04-28T12:43:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.156+0000 7fc503e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc504145330 0x7fc5040b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.156+0000 7fc503e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc504145330 0x7fc5040b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33898/0 (socket says 172.21.15.131:33898) 2024-04-28T12:43:20.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.156+0000 7fc503e00640 1 -- 172.21.15.131:0/2075066703 learned_addr learned my addr 172.21.15.131:0/2075066703 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:20.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.156+0000 7fc503400640 1 --1- 172.21.15.131:0/2075066703 >> v1:172.21.15.131:6789/0 conn(0x7fc5041628e0 0x7fc5040af720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33200/0 (socket says 172.21.15.131:33200) 2024-04-28T12:43:20.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.157+0000 7fc503e00640 1 -- 172.21.15.131:0/2075066703 >> v1:172.21.15.157:6789/0 conn(0x7fc5040afe30 legacy=0x7fc504073000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.157+0000 7fc503e00640 1 -- 172.21.15.131:0/2075066703 >> v1:172.21.15.131:6789/0 conn(0x7fc5041628e0 legacy=0x7fc5040af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.157+0000 7fc503e00640 1 -- 172.21.15.131:0/2075066703 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5040ae960 con 0x7fc504145330 2024-04-28T12:43:20.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.157+0000 7fc502a00640 1 -- 172.21.15.131:0/2075066703 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2955743756 0 0) 0x7fc504145710 con 0x7fc5041628e0 2024-04-28T12:43:20.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.158+0000 7fc503e00640 1 --2- 172.21.15.131:0/2075066703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc504145330 0x7fc5040b3940 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc5040b03d0 tx=0x7fc4f402a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d7c23f06c82c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:20.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.158+0000 7fc502a00640 1 -- 172.21.15.131:0/2075066703 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4037070 con 0x7fc504145330 2024-04-28T12:43:20.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.158+0000 7fc502a00640 1 -- 172.21.15.131:0/2075066703 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f4032640 con 0x7fc504145330 2024-04-28T12:43:20.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.158+0000 7fc502a00640 1 -- 172.21.15.131:0/2075066703 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4032a10 con 0x7fc504145330 2024-04-28T12:43:20.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.159+0000 7fc509600640 1 -- 172.21.15.131:0/2075066703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc504145330 msgr2=0x7fc5040b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.159+0000 7fc509600640 1 --2- 172.21.15.131:0/2075066703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc504145330 0x7fc5040b3940 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc5040b03d0 tx=0x7fc4f402a2e0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.159+0000 7fc509600640 1 -- 172.21.15.131:0/2075066703 shutdown_connections 2024-04-28T12:43:20.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.159+0000 7fc509600640 1 --2- 172.21.15.131:0/2075066703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc504145330 0x7fc5040b3940 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.159+0000 7fc509600640 1 -- 172.21.15.131:0/2075066703 >> 172.21.15.131:0/2075066703 conn(0x7fc504068090 msgr2=0x7fc504068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:20.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.159+0000 7fc509600640 1 -- 172.21.15.131:0/2075066703 shutdown_connections 2024-04-28T12:43:20.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.159+0000 7fc509600640 1 -- 172.21.15.131:0/2075066703 wait complete. 2024-04-28T12:43:20.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.161+0000 7fc509600640 1 Processor -- start 2024-04-28T12:43:20.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.161+0000 7fc509600640 1 -- start start 2024-04-28T12:43:20.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.162+0000 7fc509600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5040afe30 0x7fc5040b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.162+0000 7fc509600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc504145330 0x7fc5040cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.162+0000 7fc509600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5041628e0 0x7fc5040b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.162+0000 7fc509600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc5040af200 con 0x7fc5040afe30 2024-04-28T12:43:20.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.162+0000 7fc509600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5040aef20 con 0x7fc504145330 2024-04-28T12:43:20.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.162+0000 7fc509600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc5040b3e80 con 0x7fc5041628e0 2024-04-28T12:43:20.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.163+0000 7fc503e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5040afe30 0x7fc5040b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.163+0000 7fc503e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5040afe30 0x7fc5040b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33912/0 (socket says 172.21.15.131:33912) 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.163+0000 7fc503e00640 1 -- 172.21.15.131:0/397906566 learned_addr learned my addr 172.21.15.131:0/397906566 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.163+0000 7fc508c00640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5041628e0 0x7fc5040b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.163+0000 7fc503400640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc504145330 0x7fc5040cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.163+0000 7fc503e00640 1 -- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5041628e0 msgr2=0x7fc5040b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.163+0000 7fc503e00640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5041628e0 0x7fc5040b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.163+0000 7fc503e00640 1 -- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc504145330 msgr2=0x7fc5040cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.163+0000 7fc503e00640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc504145330 0x7fc5040cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.163+0000 7fc503e00640 1 -- 172.21.15.131:0/397906566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5040b2fc0 con 0x7fc5040afe30 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.163+0000 7fc503400640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc504145330 0x7fc5040cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.163+0000 7fc503e00640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5040afe30 0x7fc5040b2890 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc4f402a320 tx=0x7fc4f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.164+0000 7fc4fbe00640 1 -- 172.21.15.131:0/397906566 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4037040 con 0x7fc5040afe30 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.164+0000 7fc4fbe00640 1 -- 172.21.15.131:0/397906566 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f4042070 con 0x7fc5040afe30 2024-04-28T12:43:20.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.164+0000 7fc4fbe00640 1 -- 172.21.15.131:0/397906566 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4030bb0 con 0x7fc5040afe30 2024-04-28T12:43:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.164+0000 7fc509600640 1 -- 172.21.15.131:0/397906566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5040b32a0 con 0x7fc5040afe30 2024-04-28T12:43:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.164+0000 7fc509600640 1 -- 172.21.15.131:0/397906566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5040b0a10 con 0x7fc5040afe30 2024-04-28T12:43:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.165+0000 7fc509600640 1 -- 172.21.15.131:0/397906566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc504069350 con 0x7fc5040afe30 2024-04-28T12:43:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.166+0000 7fc4fbe00640 1 -- 172.21.15.131:0/397906566 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4f4032500 con 0x7fc5040afe30 2024-04-28T12:43:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.167+0000 7fc4fbe00640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4e406c8f0 0x7fc4e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.167+0000 7fc4fbe00640 1 -- 172.21.15.131:0/397906566 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fc4f40ac5c0 con 0x7fc5040afe30 2024-04-28T12:43:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.167+0000 7fc503400640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4e406c8f0 0x7fc4e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.168+0000 7fc503400640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4e406c8f0 0x7fc4e406edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc504140690 tx=0x7fc4f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:20.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.171+0000 7fc4fbe00640 1 -- 172.21.15.131:0/397906566 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4f407aac0 con 0x7fc5040afe30 2024-04-28T12:43:20.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.370+0000 7fc509600640 1 -- 172.21.15.131:0/397906566 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fc5040b25b0 con 0x7fc4e406c8f0 2024-04-28T12:43:20.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.371+0000 7fc4fbe00640 1 -- 172.21.15.131:0/397906566 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc5040b25b0 con 0x7fc4e406c8f0 2024-04-28T12:43:20.368 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 0 to deep-scrub 2024-04-28T12:43:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.374+0000 7fc509600640 1 -- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4e406c8f0 msgr2=0x7fc4e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.374+0000 7fc509600640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4e406c8f0 0x7fc4e406edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc504140690 tx=0x7fc4f0009040 comp rx=0 tx=0).stop 2024-04-28T12:43:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.374+0000 7fc509600640 1 -- 172.21.15.131:0/397906566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5040afe30 msgr2=0x7fc5040b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.374+0000 7fc509600640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5040afe30 0x7fc5040b2890 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc4f402a320 tx=0x7fc4f40046c0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.374+0000 7fc509600640 1 -- 172.21.15.131:0/397906566 shutdown_connections 2024-04-28T12:43:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.374+0000 7fc509600640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc4e406c8f0 0x7fc4e406edb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.374+0000 7fc509600640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5041628e0 0x7fc5040b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.374+0000 7fc509600640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc504145330 0x7fc5040cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.374+0000 7fc509600640 1 --2- 172.21.15.131:0/397906566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5040afe30 0x7fc5040b2890 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.374+0000 7fc509600640 1 -- 172.21.15.131:0/397906566 >> 172.21.15.131:0/397906566 conn(0x7fc504068090 msgr2=0x7fc50406ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:20.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.375+0000 7fc509600640 1 -- 172.21.15.131:0/397906566 shutdown_connections 2024-04-28T12:43:20.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.375+0000 7fc509600640 1 -- 172.21.15.131:0/397906566 wait complete. 2024-04-28T12:43:20.388 INFO:tasks.ceph:Scrubbing osd.1 2024-04-28T12:43:20.388 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-28T12:43:20.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.600+0000 7fe3b7200640 1 Processor -- start 2024-04-28T12:43:20.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.600+0000 7fe3b7200640 1 -- start start 2024-04-28T12:43:20.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.602+0000 7fe3b7200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fe3b80b52d0 0x7fe3b8071c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.602+0000 7fe3b7200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b80738c0 con 0x7fe3b8145450 2024-04-28T12:43:20.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.602+0000 7fe3b7200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b8073e40 con 0x7fe3b806f6c0 2024-04-28T12:43:20.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.602+0000 7fe3b7200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b806fb40 con 0x7fe3b80b52d0 2024-04-28T12:43:20.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.602+0000 7fe3b5e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe3b8145450 0x7fe3b8160090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33216/0 (socket says 172.21.15.131:33216) 2024-04-28T12:43:20.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.602+0000 7fe3b5e00640 1 -- 172.21.15.131:0/1844796307 learned_addr learned my addr 172.21.15.131:0/1844796307 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:20.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 880462951 0 0) 0x7fe3b80738c0 con 0x7fe3b8145450 2024-04-28T12:43:20.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe39c003520 con 0x7fe3b8145450 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b6800640 1 --2- 172.21.15.131:0/1844796307 >> v2:172.21.15.157:3300/0 conn(0x7fe3b80b52d0 0x7fe3b8071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1519703519 0 0) 0x7fe39c003520 con 0x7fe3b8145450 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 >> v2:172.21.15.157:3300/0 conn(0x7fe3b80b52d0 msgr2=0x7fe3b8071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 --2- 172.21.15.131:0/1844796307 >> v2:172.21.15.157:3300/0 conn(0x7fe3b80b52d0 0x7fe3b8071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 >> v1:172.21.15.157:6789/0 conn(0x7fe3b806f6c0 legacy=0x7fe3b80b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b8073b60 con 0x7fe3b8145450 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fe3a80019f0 con 0x7fe3b8145450 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b6800640 1 --2- 172.21.15.131:0/1844796307 >> v2:172.21.15.157:3300/0 conn(0x7fe3b80b52d0 0x7fe3b8071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 >> v1:172.21.15.131:6789/0 conn(0x7fe3b8145450 legacy=0x7fe3b8160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe39c003a00 0x7fe39c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39c004800 0x7fe39c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe39c0052e0 0x7fe39c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b806fb40 con 0x7fe39c0052e0 2024-04-28T12:43:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c00c230 con 0x7fe39c003a00 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.603+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe39c00e640 con 0x7fe39c004800 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.604+0000 7fe3b6800640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe39c0052e0 0x7fe39c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.604+0000 7fe3b5e00640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe39c003a00 0x7fe39c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.604+0000 7fe3b6800640 1 -- 172.21.15.131:0/1844796307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39c004800 msgr2=0x7fe39c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.604+0000 7fe3ad400640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39c004800 0x7fe39c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.604+0000 7fe3b6800640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39c004800 0x7fe39c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.604+0000 7fe3b6800640 1 -- 172.21.15.131:0/1844796307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe39c003a00 msgr2=0x7fe39c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.604+0000 7fe3b6800640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe39c003a00 0x7fe39c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.604+0000 7fe3b6800640 1 -- 172.21.15.131:0/1844796307 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c003520 con 0x7fe39c0052e0 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.604+0000 7fe3b5e00640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe39c003a00 0x7fe39c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.604+0000 7fe3ad400640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39c004800 0x7fe39c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.605+0000 7fe3b6800640 1 -- 172.21.15.131:0/1844796307 reap_dead start 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.605+0000 7fe3b6800640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe39c0052e0 0x7fe39c00b9c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80737f0 tx=0x7fe39c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41225485655a65b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:20.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.605+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0037070 con 0x7fe39c0052e0 2024-04-28T12:43:20.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.605+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b002ac20 con 0x7fe39c0052e0 2024-04-28T12:43:20.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.605+0000 7fe3b5400640 1 -- 172.21.15.131:0/1844796307 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000a650 con 0x7fe39c0052e0 2024-04-28T12:43:20.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.606+0000 7fe3b7200640 1 -- 172.21.15.131:0/1844796307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe39c0052e0 msgr2=0x7fe39c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.606+0000 7fe3b7200640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe39c0052e0 0x7fe39c00b9c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80737f0 tx=0x7fe39c005830 comp rx=0 tx=0).stop 2024-04-28T12:43:20.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.606+0000 7fe3b7200640 1 -- 172.21.15.131:0/1844796307 shutdown_connections 2024-04-28T12:43:20.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.606+0000 7fe3b7200640 1 --2- 172.21.15.131:0/1844796307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe39c0052e0 0x7fe39c00b9c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.606+0000 7fe3b7200640 1 -- 172.21.15.131:0/1844796307 >> 172.21.15.131:0/1844796307 conn(0x7fe3b80b26b0 msgr2=0x7fe3b80b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:20.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.606+0000 7fe3b7200640 1 -- 172.21.15.131:0/1844796307 shutdown_connections 2024-04-28T12:43:20.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.606+0000 7fe3b7200640 1 -- 172.21.15.131:0/1844796307 wait complete. 2024-04-28T12:43:20.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.608+0000 7fe3b7200640 1 Processor -- start 2024-04-28T12:43:20.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.608+0000 7fe3b7200640 1 -- start start 2024-04-28T12:43:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.609+0000 7fe3b7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39c0052e0 0x7fe3b81f0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.609+0000 7fe3b7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b815e420 0x7fe3b813cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.609+0000 7fe3b7200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3b813a2c0 0x7fe3b813a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.609+0000 7fe3b7200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b80cd4e0 con 0x7fe3b813a2c0 2024-04-28T12:43:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.609+0000 7fe3b7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b806eec0 con 0x7fe3b815e420 2024-04-28T12:43:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.609+0000 7fe3b7200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b80cd660 con 0x7fe39c0052e0 2024-04-28T12:43:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.609+0000 7fe3b6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3b813a2c0 0x7fe3b813a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.609+0000 7fe3b6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3b813a2c0 0x7fe3b813a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33924/0 (socket says 172.21.15.131:33924) 2024-04-28T12:43:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.609+0000 7fe3b6800640 1 -- 172.21.15.131:0/3057440744 learned_addr learned my addr 172.21.15.131:0/3057440744 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.610+0000 7fe3b5e00640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39c0052e0 0x7fe3b81f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.610+0000 7fe3b6800640 1 -- 172.21.15.131:0/3057440744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39c0052e0 msgr2=0x7fe3b81f0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.610+0000 7fe3b6800640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe39c0052e0 0x7fe3b81f0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.610+0000 7fe3b6800640 1 -- 172.21.15.131:0/3057440744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b815e420 msgr2=0x7fe3b813cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.610+0000 7fe3b6800640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3b815e420 0x7fe3b813cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.610+0000 7fe3b6800640 1 -- 172.21.15.131:0/3057440744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b815d380 con 0x7fe3b813a2c0 2024-04-28T12:43:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.610+0000 7fe3b6800640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3b813a2c0 0x7fe3b813a740 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000ab40 tx=0x7fe3b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.610+0000 7fe3aea00640 1 -- 172.21.15.131:0/3057440744 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0037070 con 0x7fe3b813a2c0 2024-04-28T12:43:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.610+0000 7fe3aea00640 1 -- 172.21.15.131:0/3057440744 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b0042070 con 0x7fe3b813a2c0 2024-04-28T12:43:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.610+0000 7fe3aea00640 1 -- 172.21.15.131:0/3057440744 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000a650 con 0x7fe3b813a2c0 2024-04-28T12:43:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.610+0000 7fe3b7200640 1 -- 172.21.15.131:0/3057440744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b815d660 con 0x7fe3b813a2c0 2024-04-28T12:43:20.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.610+0000 7fe3b7200640 1 -- 172.21.15.131:0/3057440744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b815dc20 con 0x7fe3b813a2c0 2024-04-28T12:43:20.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.611+0000 7fe3b7200640 1 -- 172.21.15.131:0/3057440744 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe3b815d130 con 0x7fe3b813a2c0 2024-04-28T12:43:20.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.612+0000 7fe3aea00640 1 -- 172.21.15.131:0/3057440744 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3b003c830 con 0x7fe3b813a2c0 2024-04-28T12:43:20.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.613+0000 7fe3aea00640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe39406c8f0 0x7fe39406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.613+0000 7fe3aea00640 1 -- 172.21.15.131:0/3057440744 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fe3b00aba10 con 0x7fe3b813a2c0 2024-04-28T12:43:20.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.614+0000 7fe3aea00640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] conn(0x7fe394072d90 0x7fe3940751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.614+0000 7fe3aea00640 1 -- 172.21.15.131:0/3057440744 --> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3b002cea0 con 0x7fe394072d90 2024-04-28T12:43:20.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.614+0000 7fe3aea00640 1 -- 172.21.15.131:0/3057440744 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe3b00b02c0 con 0x7fe3b813a2c0 2024-04-28T12:43:20.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.614+0000 7fe3ad400640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] conn(0x7fe394072d90 0x7fe3940751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.614+0000 7fe3b5e00640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe39406c8f0 0x7fe39406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.615+0000 7fe3ad400640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] conn(0x7fe394072d90 0x7fe3940751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:20.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.615+0000 7fe3b5e00640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe39406c8f0 0x7fe39406edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8001350 tx=0x7fe3a8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:20.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.615+0000 7fe3aea00640 1 -- 172.21.15.131:0/3057440744 <== osd.1 v2:172.21.15.131:6800/703178662 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3b002cea0 con 0x7fe394072d90 2024-04-28T12:43:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.630+0000 7fe3b7200640 1 -- 172.21.15.131:0/3057440744 --> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe3b8069350 con 0x7fe394072d90 2024-04-28T12:43:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.630+0000 7fe3aea00640 1 -- 172.21.15.131:0/3057440744 <== osd.1 v2:172.21.15.131:6800/703178662 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe3b8069350 con 0x7fe394072d90 2024-04-28T12:43:20.626 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:43:20.626 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T12:43:20.626 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:43:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.630+0000 7fe3b7200640 1 -- 172.21.15.131:0/3057440744 >> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] conn(0x7fe394072d90 msgr2=0x7fe3940751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.630+0000 7fe3b7200640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.131:6800/703178662,v1:172.21.15.131:6801/703178662] conn(0x7fe394072d90 0x7fe3940751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.631+0000 7fe3b7200640 1 -- 172.21.15.131:0/3057440744 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe39406c8f0 msgr2=0x7fe39406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.631+0000 7fe3b7200640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fe39406c8f0 0x7fe39406edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8001350 tx=0x7fe3a8009040 comp rx=0 tx=0).stop 2024-04-28T12:43:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.631+0000 7fe3b7200640 1 -- 172.21.15.131:0/3057440744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3b813a2c0 msgr2=0x7fe3b813a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.631+0000 7fe3b7200640 1 --2- 172.21.15.131:0/3057440744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3b813a2c0 0x7fe3b813a740 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000ab40 tx=0x7fe3b0004730 comp rx=0 tx=0).stop 2024-04-28T12:43:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.631+0000 7fe3b6800640 1 -- 172.21.15.131:0/3057440744 reap_dead start 2024-04-28T12:43:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.631+0000 7fe3b7200640 1 -- 172.21.15.131:0/3057440744 shutdown_connections 2024-04-28T12:43:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.631+0000 7fe3b7200640 1 -- 172.21.15.131:0/3057440744 >> 172.21.15.131:0/3057440744 conn(0x7fe3b80b26b0 msgr2=0x7fe3b8072550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.631+0000 7fe3b7200640 1 -- 172.21.15.131:0/3057440744 shutdown_connections 2024-04-28T12:43:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.631+0000 7fe3b7200640 1 -- 172.21.15.131:0/3057440744 wait complete. 2024-04-28T12:43:20.643 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-28T12:43:20.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.837+0000 7f953b800640 1 Processor -- start 2024-04-28T12:43:20.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.837+0000 7f953b800640 1 -- start start 2024-04-28T12:43:20.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.839+0000 7f953b800640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9534159050 0x7f95340b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.839+0000 7f953b800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95340b3c00 0x7f95340b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.839+0000 7f953b800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9534159430 con 0x7f95340d1440 2024-04-28T12:43:20.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.839+0000 7f953b800640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f95340b5dd0 con 0x7f9534159050 2024-04-28T12:43:20.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.839+0000 7f953b800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9534073c10 con 0x7f95340b3c00 2024-04-28T12:43:20.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.839+0000 7f952be00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f95340d1440 0x7f953415efd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39604/0 (socket says 172.21.15.131:39604) 2024-04-28T12:43:20.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.839+0000 7f952be00640 1 -- 172.21.15.131:0/3926174400 learned_addr learned my addr 172.21.15.131:0/3926174400 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:20.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9539800640 1 --2- 172.21.15.131:0/3926174400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95340b3c00 0x7f95340b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9538e00640 1 --2- 172.21.15.131:0/3926174400 >> v2:172.21.15.157:3300/0 conn(0x7f9534159050 0x7f95340b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3328365828 0 0) 0x7f9534159430 con 0x7f95340d1440 2024-04-28T12:43:20.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9518005730 con 0x7f95340d1440 2024-04-28T12:43:20.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4242386296 0 0) 0x7f9518005730 con 0x7f95340d1440 2024-04-28T12:43:20.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95340b3c00 msgr2=0x7f95340b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:20.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 --2- 172.21.15.131:0/3926174400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95340b3c00 0x7f95340b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 >> v2:172.21.15.157:3300/0 conn(0x7f9534159050 msgr2=0x7f95340b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:20.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 --2- 172.21.15.131:0/3926174400 >> v2:172.21.15.157:3300/0 conn(0x7f9534159050 0x7f95340b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9534073480 con 0x7f95340d1440 2024-04-28T12:43:20.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f951c001670 con 0x7f95340d1440 2024-04-28T12:43:20.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9538e00640 1 --2- 172.21.15.131:0/3926174400 >> v2:172.21.15.157:3300/0 conn(0x7f9534159050 0x7f95340b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:43:20.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 >> v1:172.21.15.131:6789/0 conn(0x7f95340d1440 legacy=0x7f953415efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 --2- 172.21.15.131:0/3926174400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95340b3c00 0x7f9518005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 --2- 172.21.15.131:0/3926174400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9518006140 0x7f9518006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 --2- 172.21.15.131:0/3926174400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9518006f50 0x7f95180073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9534073c10 con 0x7f9518006140 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f951800dbd0 con 0x7f9518006f50 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.840+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f951800dda0 con 0x7f95340b3c00 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.841+0000 7f952be00640 1 --2- 172.21.15.131:0/3926174400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9518006140 0x7f9518006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.841+0000 7f9538e00640 1 --2- 172.21.15.131:0/3926174400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95340b3c00 0x7f9518005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.841+0000 7f9539800640 1 --2- 172.21.15.131:0/3926174400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9518006f50 0x7f95180073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.841+0000 7f952be00640 1 -- 172.21.15.131:0/3926174400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95340b3c00 msgr2=0x7f9518005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.841+0000 7f952be00640 1 --2- 172.21.15.131:0/3926174400 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95340b3c00 0x7f9518005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.841+0000 7f952be00640 1 -- 172.21.15.131:0/3926174400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9518006f50 msgr2=0x7f95180073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.841+0000 7f952be00640 1 --2- 172.21.15.131:0/3926174400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9518006f50 0x7f95180073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.841+0000 7f952be00640 1 -- 172.21.15.131:0/3926174400 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9518005730 con 0x7f9518006140 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.842+0000 7f952be00640 1 --2- 172.21.15.131:0/3926174400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9518006140 0x7f9518006a10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f951c00c210 tx=0x7f9518007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212c89116103f73d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.842+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c007d50 con 0x7f9518006140 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.842+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951c005af0 con 0x7f9518006140 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.842+0000 7f9533e00640 1 -- 172.21.15.131:0/3926174400 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c007070 con 0x7f9518006140 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.843+0000 7f953b800640 1 -- 172.21.15.131:0/3926174400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9518006140 msgr2=0x7f9518006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.843+0000 7f953b800640 1 --2- 172.21.15.131:0/3926174400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9518006140 0x7f9518006a10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f951c00c210 tx=0x7f9518007a40 comp rx=0 tx=0).stop 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.843+0000 7f9539800640 1 -- 172.21.15.131:0/3926174400 reap_dead start 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.843+0000 7f953b800640 1 -- 172.21.15.131:0/3926174400 shutdown_connections 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.843+0000 7f953b800640 1 -- 172.21.15.131:0/3926174400 >> 172.21.15.131:0/3926174400 conn(0x7f95340b2900 msgr2=0x7f95340b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.843+0000 7f953b800640 1 -- 172.21.15.131:0/3926174400 shutdown_connections 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.843+0000 7f953b800640 1 -- 172.21.15.131:0/3926174400 wait complete. 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.845+0000 7f953b800640 1 Processor -- start 2024-04-28T12:43:20.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.845+0000 7f953b800640 1 -- start start 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.846+0000 7f953b800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9518006140 0x7f953414de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.846+0000 7f953b800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f953414e370 0x7f9534153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.846+0000 7f953b800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f953414cf90 0x7f9534153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.846+0000 7f953b800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9534071f50 con 0x7f953414cf90 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.846+0000 7f953b800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9534073e60 con 0x7f953414e370 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.846+0000 7f953b800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9534074140 con 0x7f9518006140 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.846+0000 7f9539800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f953414cf90 0x7f9534153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.846+0000 7f9539800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f953414cf90 0x7f9534153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48650/0 (socket says 172.21.15.131:48650) 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.846+0000 7f9539800640 1 -- 172.21.15.131:0/2612744598 learned_addr learned my addr 172.21.15.131:0/2612744598 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.846+0000 7f9538e00640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9518006140 0x7f953414de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f952be00640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f953414e370 0x7f9534153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f9539800640 1 -- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9518006140 msgr2=0x7f953414de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f9539800640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9518006140 0x7f953414de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f9539800640 1 -- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f953414e370 msgr2=0x7f9534153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f9539800640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f953414e370 0x7f9534153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f9539800640 1 -- 172.21.15.131:0/2612744598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95341f52b0 con 0x7f953414cf90 2024-04-28T12:43:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f952be00640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f953414e370 0x7f9534153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f9539800640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f953414cf90 0x7f9534153750 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f953406d150 tx=0x7f952c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f9531600640 1 -- 172.21.15.131:0/2612744598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c01c070 con 0x7f953414cf90 2024-04-28T12:43:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f9531600640 1 -- 172.21.15.131:0/2612744598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c017400 con 0x7f953414cf90 2024-04-28T12:43:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f9531600640 1 -- 172.21.15.131:0/2612744598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c0177f0 con 0x7f953414cf90 2024-04-28T12:43:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f953b800640 1 -- 172.21.15.131:0/2612744598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95341f5590 con 0x7f953414cf90 2024-04-28T12:43:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.847+0000 7f953b800640 1 -- 172.21.15.131:0/2612744598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95341f5b50 con 0x7f953414cf90 2024-04-28T12:43:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.849+0000 7f953b800640 1 -- 172.21.15.131:0/2612744598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9534069350 con 0x7f953414cf90 2024-04-28T12:43:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.850+0000 7f9531600640 1 -- 172.21.15.131:0/2612744598 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f952c01f070 con 0x7f953414cf90 2024-04-28T12:43:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.850+0000 7f9531600640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f951006c8f0 0x7f951006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:20.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.851+0000 7f9531600640 1 -- 172.21.15.131:0/2612744598 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f952c08ba10 con 0x7f953414cf90 2024-04-28T12:43:20.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.851+0000 7f9538e00640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f951006c8f0 0x7f951006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:20.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.852+0000 7f9538e00640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f951006c8f0 0x7f951006edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9524006fd0 tx=0x7f9524008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:20.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:20.855+0000 7f9531600640 1 -- 172.21.15.131:0/2612744598 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f952c012070 con 0x7f953414cf90 2024-04-28T12:43:21.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.059+0000 7f953b800640 1 -- 172.21.15.131:0/2612744598 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f953406fcd0 con 0x7f951006c8f0 2024-04-28T12:43:21.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.060+0000 7f9531600640 1 -- 172.21.15.131:0/2612744598 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f953406fcd0 con 0x7f951006c8f0 2024-04-28T12:43:21.057 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 1 to deep-scrub 2024-04-28T12:43:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.063+0000 7f953b800640 1 -- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f951006c8f0 msgr2=0x7f951006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.063+0000 7f953b800640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f951006c8f0 0x7f951006edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9524006fd0 tx=0x7f9524008040 comp rx=0 tx=0).stop 2024-04-28T12:43:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.063+0000 7f953b800640 1 -- 172.21.15.131:0/2612744598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f953414cf90 msgr2=0x7f9534153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.063+0000 7f953b800640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f953414cf90 0x7f9534153750 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f953406d150 tx=0x7f952c00f4a0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.063+0000 7f953b800640 1 -- 172.21.15.131:0/2612744598 shutdown_connections 2024-04-28T12:43:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.063+0000 7f953b800640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f951006c8f0 0x7f951006edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.063+0000 7f953b800640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f953414cf90 0x7f9534153750 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.063+0000 7f953b800640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f953414e370 0x7f9534153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.063+0000 7f953b800640 1 --2- 172.21.15.131:0/2612744598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9518006140 0x7f953414de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.063+0000 7f953b800640 1 -- 172.21.15.131:0/2612744598 >> 172.21.15.131:0/2612744598 conn(0x7f95340b2900 msgr2=0x7f953406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.063+0000 7f953b800640 1 -- 172.21.15.131:0/2612744598 shutdown_connections 2024-04-28T12:43:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.063+0000 7f953b800640 1 -- 172.21.15.131:0/2612744598 wait complete. 2024-04-28T12:43:21.077 INFO:tasks.ceph:Scrubbing osd.2 2024-04-28T12:43:21.077 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-28T12:43:21.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.272+0000 7fc656400640 1 Processor -- start 2024-04-28T12:43:21.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.273+0000 7fc656400640 1 -- start start 2024-04-28T12:43:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.274+0000 7fc656400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6500cf510 0x7fc6500d3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.274+0000 7fc656400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc65006d520 0x7fc6500d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.274+0000 7fc656400640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc65006da50 con 0x7fc65015f380 2024-04-28T12:43:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.274+0000 7fc656400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc6500d0d40 con 0x7fc65006d520 2024-04-28T12:43:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.274+0000 7fc656400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc6500d18a0 con 0x7fc6500cf510 2024-04-28T12:43:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.275+0000 7fc655a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc65006d520 0x7fc6500d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.275+0000 7fc655a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc65006d520 0x7fc6500d1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48662/0 (socket says 172.21.15.131:48662) 2024-04-28T12:43:21.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.275+0000 7fc655a00640 1 -- 172.21.15.131:0/1316514318 learned_addr learned my addr 172.21.15.131:0/1316514318 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.275+0000 7fc655000640 1 --2- 172.21.15.131:0/1316514318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6500cf510 0x7fc6500d3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.275+0000 7fc655a00640 1 -- 172.21.15.131:0/1316514318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6500cf510 msgr2=0x7fc6500d3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.275+0000 7fc655a00640 1 --2- 172.21.15.131:0/1316514318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6500cf510 0x7fc6500d3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.275+0000 7fc655a00640 1 -- 172.21.15.131:0/1316514318 >> v1:172.21.15.157:6789/0 conn(0x7fc65015f380 legacy=0x7fc6500cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:21.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.275+0000 7fc655a00640 1 -- 172.21.15.131:0/1316514318 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6500d0f70 con 0x7fc65006d520 2024-04-28T12:43:21.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.276+0000 7fc655a00640 1 --2- 172.21.15.131:0/1316514318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc65006d520 0x7fc6500d1d30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc648009870 tx=0x7fc64802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd37ded5896632e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:21.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.277+0000 7fc64f400640 1 -- 172.21.15.131:0/1316514318 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648037070 con 0x7fc65006d520 2024-04-28T12:43:21.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.277+0000 7fc64f400640 1 -- 172.21.15.131:0/1316514318 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc648032440 con 0x7fc65006d520 2024-04-28T12:43:21.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.277+0000 7fc64f400640 1 -- 172.21.15.131:0/1316514318 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648032810 con 0x7fc65006d520 2024-04-28T12:43:21.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.277+0000 7fc656400640 1 -- 172.21.15.131:0/1316514318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc65006d520 msgr2=0x7fc6500d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.277+0000 7fc656400640 1 --2- 172.21.15.131:0/1316514318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc65006d520 0x7fc6500d1d30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc648009870 tx=0x7fc64802a210 comp rx=0 tx=0).stop 2024-04-28T12:43:21.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.278+0000 7fc656400640 1 -- 172.21.15.131:0/1316514318 shutdown_connections 2024-04-28T12:43:21.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.278+0000 7fc656400640 1 --2- 172.21.15.131:0/1316514318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc65006d520 0x7fc6500d1d30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.278+0000 7fc656400640 1 --2- 172.21.15.131:0/1316514318 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc6500cf510 0x7fc6500d3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.278+0000 7fc656400640 1 -- 172.21.15.131:0/1316514318 >> 172.21.15.131:0/1316514318 conn(0x7fc6500d2b00 msgr2=0x7fc6500d2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:21.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.278+0000 7fc656400640 1 -- 172.21.15.131:0/1316514318 shutdown_connections 2024-04-28T12:43:21.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.278+0000 7fc656400640 1 -- 172.21.15.131:0/1316514318 wait complete. 2024-04-28T12:43:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.279+0000 7fc656400640 1 Processor -- start 2024-04-28T12:43:21.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.280+0000 7fc656400640 1 -- start start 2024-04-28T12:43:21.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.280+0000 7fc656400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65006d520 0x7fc650158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.280+0000 7fc656400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cf510 0x7fc6500c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc656400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65015f380 0x7fc6500b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc656400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500b1f20 con 0x7fc6500cf510 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc656400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500d1530 con 0x7fc65006d520 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc656400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc6500cd150 con 0x7fc65015f380 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc64fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cf510 0x7fc6500c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc64fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cf510 0x7fc6500c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48672/0 (socket says 172.21.15.131:48672) 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc64fe00640 1 -- 172.21.15.131:0/199038666 learned_addr learned my addr 172.21.15.131:0/199038666 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc655000640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65006d520 0x7fc650158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc655a00640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65015f380 0x7fc6500b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc64fe00640 1 -- 172.21.15.131:0/199038666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65015f380 msgr2=0x7fc6500b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc64fe00640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc65015f380 0x7fc6500b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc64fe00640 1 -- 172.21.15.131:0/199038666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65006d520 msgr2=0x7fc650158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc64fe00640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc65006d520 0x7fc650158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.281+0000 7fc64fe00640 1 -- 172.21.15.131:0/199038666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6500ca140 con 0x7fc6500cf510 2024-04-28T12:43:21.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.282+0000 7fc64fe00640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cf510 0x7fc6500c8cb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc63800b3f0 tx=0x7fc63800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:21.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.282+0000 7fc64cc00640 1 -- 172.21.15.131:0/199038666 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638015020 con 0x7fc6500cf510 2024-04-28T12:43:21.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.282+0000 7fc64cc00640 1 -- 172.21.15.131:0/199038666 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc638011830 con 0x7fc6500cf510 2024-04-28T12:43:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.282+0000 7fc656400640 1 -- 172.21.15.131:0/199038666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6500ca420 con 0x7fc6500cf510 2024-04-28T12:43:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.282+0000 7fc64cc00640 1 -- 172.21.15.131:0/199038666 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638011c20 con 0x7fc6500cf510 2024-04-28T12:43:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.282+0000 7fc656400640 1 -- 172.21.15.131:0/199038666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6500ca9e0 con 0x7fc6500cf510 2024-04-28T12:43:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.283+0000 7fc656400640 1 -- 172.21.15.131:0/199038666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc6500cadb0 con 0x7fc6500cf510 2024-04-28T12:43:21.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.284+0000 7fc64cc00640 1 -- 172.21.15.131:0/199038666 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc638018020 con 0x7fc6500cf510 2024-04-28T12:43:21.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.285+0000 7fc64cc00640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc62c06c8f0 0x7fc62c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.285+0000 7fc64cc00640 1 -- 172.21.15.131:0/199038666 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fc63808b610 con 0x7fc6500cf510 2024-04-28T12:43:21.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.285+0000 7fc64cc00640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] conn(0x7fc62c072d90 0x7fc62c0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.285+0000 7fc64cc00640 1 -- 172.21.15.131:0/199038666 --> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc638001d30 con 0x7fc62c072d90 2024-04-28T12:43:21.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.285+0000 7fc64cc00640 1 -- 172.21.15.131:0/199038666 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc638092050 con 0x7fc6500cf510 2024-04-28T12:43:21.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.286+0000 7fc655a00640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] conn(0x7fc62c072d90 0x7fc62c0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.286+0000 7fc655000640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc62c06c8f0 0x7fc62c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.286+0000 7fc655a00640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] conn(0x7fc62c072d90 0x7fc62c0751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.287+0000 7fc655000640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc62c06c8f0 0x7fc62c06edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc6501f8830 tx=0x7fc6400039b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:21.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.287+0000 7fc64cc00640 1 -- 172.21.15.131:0/199038666 <== osd.2 v2:172.21.15.131:6808/768553345 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc638001d30 con 0x7fc62c072d90 2024-04-28T12:43:21.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.302+0000 7fc656400640 1 -- 172.21.15.131:0/199038666 --> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc6500c9440 con 0x7fc62c072d90 2024-04-28T12:43:21.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.302+0000 7fc64cc00640 1 -- 172.21.15.131:0/199038666 <== osd.2 v2:172.21.15.131:6808/768553345 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc6500c9440 con 0x7fc62c072d90 2024-04-28T12:43:21.299 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:43:21.299 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T12:43:21.299 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:43:21.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.303+0000 7fc656400640 1 -- 172.21.15.131:0/199038666 >> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] conn(0x7fc62c072d90 msgr2=0x7fc62c0751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.303+0000 7fc656400640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.131:6808/768553345,v1:172.21.15.131:6809/768553345] conn(0x7fc62c072d90 0x7fc62c0751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.303+0000 7fc656400640 1 -- 172.21.15.131:0/199038666 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc62c06c8f0 msgr2=0x7fc62c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.303+0000 7fc656400640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc62c06c8f0 0x7fc62c06edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc6501f8830 tx=0x7fc6400039b0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.303+0000 7fc656400640 1 -- 172.21.15.131:0/199038666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cf510 msgr2=0x7fc6500c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.303+0000 7fc656400640 1 --2- 172.21.15.131:0/199038666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6500cf510 0x7fc6500c8cb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc63800b3f0 tx=0x7fc63800b8b0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.303+0000 7fc655a00640 1 -- 172.21.15.131:0/199038666 reap_dead start 2024-04-28T12:43:21.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.303+0000 7fc656400640 1 -- 172.21.15.131:0/199038666 shutdown_connections 2024-04-28T12:43:21.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.303+0000 7fc656400640 1 -- 172.21.15.131:0/199038666 >> 172.21.15.131:0/199038666 conn(0x7fc6500d2b00 msgr2=0x7fc6500cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:21.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.303+0000 7fc656400640 1 -- 172.21.15.131:0/199038666 shutdown_connections 2024-04-28T12:43:21.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.303+0000 7fc656400640 1 -- 172.21.15.131:0/199038666 wait complete. 2024-04-28T12:43:21.318 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-28T12:43:21.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.508+0000 7facbf200640 1 Processor -- start 2024-04-28T12:43:21.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.509+0000 7facbf200640 1 -- start start 2024-04-28T12:43:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.510+0000 7facbf200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80726c0 0x7facb8072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.510+0000 7facbf200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7facb80afc60 0x7facb80b4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.510+0000 7facbf200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facb806f7c0 con 0x7facb80af710 2024-04-28T12:43:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.510+0000 7facbf200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7facb8072fe0 con 0x7facb80afc60 2024-04-28T12:43:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.510+0000 7facbf200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7facb8167ae0 con 0x7facb80726c0 2024-04-28T12:43:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.511+0000 7facbd400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7facb80afc60 0x7facb80b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.511+0000 7facbd400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7facb80afc60 0x7facb80b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48674/0 (socket says 172.21.15.131:48674) 2024-04-28T12:43:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.511+0000 7facbd400640 1 -- 172.21.15.131:0/4092900129 learned_addr learned my addr 172.21.15.131:0/4092900129 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.511+0000 7facbde00640 1 --2- 172.21.15.131:0/4092900129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80726c0 0x7facb8072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.511+0000 7facbd400640 1 -- 172.21.15.131:0/4092900129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80726c0 msgr2=0x7facb8072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.511+0000 7facbd400640 1 --2- 172.21.15.131:0/4092900129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80726c0 0x7facb8072aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.511+0000 7facbd400640 1 -- 172.21.15.131:0/4092900129 >> v1:172.21.15.157:6789/0 conn(0x7facb80af710 legacy=0x7facb80b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.511+0000 7facbd400640 1 -- 172.21.15.131:0/4092900129 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb806dcd0 con 0x7facb80afc60 2024-04-28T12:43:21.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.511+0000 7facbde00640 1 --2- 172.21.15.131:0/4092900129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80726c0 0x7facb8072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:21.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.512+0000 7facbd400640 1 --2- 172.21.15.131:0/4092900129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb80afc60 0x7facb80b4900 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7faca8009870 tx=0x7faca802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b21541545bb0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:21.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.512+0000 7facbca00640 1 -- 172.21.15.131:0/4092900129 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca8037070 con 0x7facb80afc60 2024-04-28T12:43:21.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.512+0000 7facbca00640 1 -- 172.21.15.131:0/4092900129 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca8032440 con 0x7facb80afc60 2024-04-28T12:43:21.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.512+0000 7facbca00640 1 -- 172.21.15.131:0/4092900129 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca8032810 con 0x7facb80afc60 2024-04-28T12:43:21.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.513+0000 7facbf200640 1 -- 172.21.15.131:0/4092900129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb80afc60 msgr2=0x7facb80b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.513+0000 7facbf200640 1 --2- 172.21.15.131:0/4092900129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb80afc60 0x7facb80b4900 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7faca8009870 tx=0x7faca802a200 comp rx=0 tx=0).stop 2024-04-28T12:43:21.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.513+0000 7facbf200640 1 -- 172.21.15.131:0/4092900129 shutdown_connections 2024-04-28T12:43:21.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.513+0000 7facbf200640 1 --2- 172.21.15.131:0/4092900129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb80afc60 0x7facb80b4900 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.513+0000 7facbf200640 1 --2- 172.21.15.131:0/4092900129 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80726c0 0x7facb8072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.513+0000 7facbf200640 1 -- 172.21.15.131:0/4092900129 >> 172.21.15.131:0/4092900129 conn(0x7facb8068090 msgr2=0x7facb8068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:21.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.513+0000 7facbf200640 1 -- 172.21.15.131:0/4092900129 shutdown_connections 2024-04-28T12:43:21.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.513+0000 7facbf200640 1 -- 172.21.15.131:0/4092900129 wait complete. 2024-04-28T12:43:21.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.515+0000 7facbf200640 1 Processor -- start 2024-04-28T12:43:21.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.515+0000 7facbf200640 1 -- start start 2024-04-28T12:43:21.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.516+0000 7facbf200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80726c0 0x7facb80cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.516+0000 7facbf200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb80af710 0x7facb81567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.516+0000 7facbf200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80afc60 0x7facb8164830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.516+0000 7facbf200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7facb8167770 con 0x7facb80af710 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.516+0000 7facbf200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7facb806e290 con 0x7facb80726c0 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.516+0000 7facbf200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7facb8163420 con 0x7facb80afc60 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.516+0000 7facbd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb80af710 0x7facb81567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.517+0000 7facbd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb80af710 0x7facb81567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48690/0 (socket says 172.21.15.131:48690) 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.517+0000 7facbd400640 1 -- 172.21.15.131:0/3522507737 learned_addr learned my addr 172.21.15.131:0/3522507737 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.517+0000 7facbde00640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80726c0 0x7facb80cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.517+0000 7facbe800640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80afc60 0x7facb8164830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.517+0000 7facbd400640 1 -- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80afc60 msgr2=0x7facb8164830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.517+0000 7facbd400640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80afc60 0x7facb8164830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.517+0000 7facbd400640 1 -- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80726c0 msgr2=0x7facb80cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.517+0000 7facbd400640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80726c0 0x7facb80cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.517+0000 7facbd400640 1 -- 172.21.15.131:0/3522507737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb815cad0 con 0x7facb80af710 2024-04-28T12:43:21.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.517+0000 7facbde00640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80726c0 0x7facb80cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:21.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.517+0000 7facbd400640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb80af710 0x7facb81567a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7faca8000c00 tx=0x7faca80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:21.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.518+0000 7facb2000640 1 -- 172.21.15.131:0/3522507737 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca8037040 con 0x7facb80af710 2024-04-28T12:43:21.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.518+0000 7facb2000640 1 -- 172.21.15.131:0/3522507737 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca8042070 con 0x7facb80af710 2024-04-28T12:43:21.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.518+0000 7facb2000640 1 -- 172.21.15.131:0/3522507737 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca8030bb0 con 0x7facb80af710 2024-04-28T12:43:21.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.518+0000 7facbf200640 1 -- 172.21.15.131:0/3522507737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb815ba40 con 0x7facb80af710 2024-04-28T12:43:21.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.518+0000 7facbf200640 1 -- 172.21.15.131:0/3522507737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb815bd10 con 0x7facb80af710 2024-04-28T12:43:21.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.519+0000 7facbf200640 1 -- 172.21.15.131:0/3522507737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb8069350 con 0x7facb80af710 2024-04-28T12:43:21.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.520+0000 7facb2000640 1 -- 172.21.15.131:0/3522507737 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faca802ad10 con 0x7facb80af710 2024-04-28T12:43:21.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.521+0000 7facb2000640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac9c06c8f0 0x7fac9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.521+0000 7facb2000640 1 -- 172.21.15.131:0/3522507737 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7faca80ac2d0 con 0x7facb80af710 2024-04-28T12:43:21.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.521+0000 7facbde00640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac9c06c8f0 0x7fac9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.522+0000 7facbde00640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac9c06c8f0 0x7fac9c06edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7facac006fd0 tx=0x7facac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:21.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.525+0000 7facb2000640 1 -- 172.21.15.131:0/3522507737 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faca8045360 con 0x7facb80af710 2024-04-28T12:43:21.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.724+0000 7facbf200640 1 -- 172.21.15.131:0/3522507737 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7facb8161950 con 0x7fac9c06c8f0 2024-04-28T12:43:21.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.725+0000 7facb2000640 1 -- 172.21.15.131:0/3522507737 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7facb8161950 con 0x7fac9c06c8f0 2024-04-28T12:43:21.721 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 2 to deep-scrub 2024-04-28T12:43:21.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.727+0000 7facbf200640 1 -- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac9c06c8f0 msgr2=0x7fac9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.727+0000 7facbf200640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac9c06c8f0 0x7fac9c06edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7facac006fd0 tx=0x7facac008040 comp rx=0 tx=0).stop 2024-04-28T12:43:21.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.728+0000 7facbf200640 1 -- 172.21.15.131:0/3522507737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb80af710 msgr2=0x7facb81567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.728+0000 7facbf200640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb80af710 0x7facb81567a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7faca8000c00 tx=0x7faca80046c0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.728+0000 7facbf200640 1 -- 172.21.15.131:0/3522507737 shutdown_connections 2024-04-28T12:43:21.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.728+0000 7facbf200640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fac9c06c8f0 0x7fac9c06edb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.728+0000 7facbf200640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facb80afc60 0x7facb8164830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.728+0000 7facbf200640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facb80af710 0x7facb81567a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.728+0000 7facbf200640 1 --2- 172.21.15.131:0/3522507737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facb80726c0 0x7facb80cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.728+0000 7facbf200640 1 -- 172.21.15.131:0/3522507737 >> 172.21.15.131:0/3522507737 conn(0x7facb8068090 msgr2=0x7facb8161c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:21.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.728+0000 7facbf200640 1 -- 172.21.15.131:0/3522507737 shutdown_connections 2024-04-28T12:43:21.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.728+0000 7facbf200640 1 -- 172.21.15.131:0/3522507737 wait complete. 2024-04-28T12:43:21.743 INFO:tasks.ceph:Scrubbing osd.3 2024-04-28T12:43:21.743 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-28T12:43:21.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.944+0000 7f9925e00640 1 Processor -- start 2024-04-28T12:43:21.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.944+0000 7f9925e00640 1 -- start start 2024-04-28T12:43:21.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.945+0000 7f9925e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9920145450 0x7f992006f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.945+0000 7f9925e00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f99200b52d0 0x7f9920071c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.945+0000 7f9925e00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99200738c0 con 0x7f9920160090 2024-04-28T12:43:21.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.945+0000 7f9925e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9920073e40 con 0x7f9920145450 2024-04-28T12:43:21.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.945+0000 7f9925e00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9920073b60 con 0x7f99200b52d0 2024-04-28T12:43:21.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.946+0000 7f9924a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9920145450 0x7f992006f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.946+0000 7f9924a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9920145450 0x7f992006f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48694/0 (socket says 172.21.15.131:48694) 2024-04-28T12:43:21.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.946+0000 7f9924a00640 1 -- 172.21.15.131:0/4098533561 learned_addr learned my addr 172.21.15.131:0/4098533561 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:21.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.946+0000 7f9924a00640 1 -- 172.21.15.131:0/4098533561 >> v2:172.21.15.157:3300/0 conn(0x7f99200b52d0 msgr2=0x7f9920071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:21.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.946+0000 7f9924a00640 1 --2- 172.21.15.131:0/4098533561 >> v2:172.21.15.157:3300/0 conn(0x7f99200b52d0 0x7f9920071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.946+0000 7f9924a00640 1 -- 172.21.15.131:0/4098533561 >> v1:172.21.15.157:6789/0 conn(0x7f9920160090 legacy=0x7f99200b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:21.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.946+0000 7f9924a00640 1 -- 172.21.15.131:0/4098533561 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992006ebe0 con 0x7f9920145450 2024-04-28T12:43:21.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.947+0000 7f9924a00640 1 --2- 172.21.15.131:0/4098533561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9920145450 0x7f992006f6c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f99200737d0 tx=0x7f991002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69356f32c18855b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:21.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.947+0000 7f991fe00640 1 -- 172.21.15.131:0/4098533561 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910037070 con 0x7f9920145450 2024-04-28T12:43:21.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.947+0000 7f991fe00640 1 -- 172.21.15.131:0/4098533561 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9910032440 con 0x7f9920145450 2024-04-28T12:43:21.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.947+0000 7f991fe00640 1 -- 172.21.15.131:0/4098533561 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910032810 con 0x7f9920145450 2024-04-28T12:43:21.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.948+0000 7f9925e00640 1 -- 172.21.15.131:0/4098533561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9920145450 msgr2=0x7f992006f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.948+0000 7f9925e00640 1 --2- 172.21.15.131:0/4098533561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9920145450 0x7f992006f6c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f99200737d0 tx=0x7f991002a230 comp rx=0 tx=0).stop 2024-04-28T12:43:21.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.948+0000 7f9925e00640 1 -- 172.21.15.131:0/4098533561 shutdown_connections 2024-04-28T12:43:21.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.948+0000 7f9925e00640 1 --2- 172.21.15.131:0/4098533561 >> v2:172.21.15.157:3300/0 conn(0x7f99200b52d0 0x7f9920071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.948+0000 7f9925e00640 1 --2- 172.21.15.131:0/4098533561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9920145450 0x7f992006f6c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.948+0000 7f9925e00640 1 -- 172.21.15.131:0/4098533561 >> 172.21.15.131:0/4098533561 conn(0x7f99200b26b0 msgr2=0x7f99200b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:21.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.948+0000 7f9925e00640 1 -- 172.21.15.131:0/4098533561 shutdown_connections 2024-04-28T12:43:21.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.948+0000 7f9925e00640 1 -- 172.21.15.131:0/4098533561 wait complete. 2024-04-28T12:43:21.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.950+0000 7f9925e00640 1 Processor -- start 2024-04-28T12:43:21.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.950+0000 7f9925e00640 1 -- start start 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.951+0000 7f9925e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99200b52d0 0x7f99201437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.951+0000 7f9925e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9920145450 0x7f99200cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.951+0000 7f9925e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9920160090 0x7f99200cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.951+0000 7f9925e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f99200cd4e0 con 0x7f99200b52d0 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.951+0000 7f9925e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9920160540 con 0x7f9920145450 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.951+0000 7f9925e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f99200cd660 con 0x7f9920160090 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.951+0000 7f9924a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99200b52d0 0x7f99201437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.951+0000 7f9924a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99200b52d0 0x7f99201437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48702/0 (socket says 172.21.15.131:48702) 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.951+0000 7f9924a00640 1 -- 172.21.15.131:0/1711279714 learned_addr learned my addr 172.21.15.131:0/1711279714 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.952+0000 7f9917e00640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9920145450 0x7f99200cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.952+0000 7f9925400640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9920160090 0x7f99200cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.952+0000 7f9924a00640 1 -- 172.21.15.131:0/1711279714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9920160090 msgr2=0x7f99200cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.952+0000 7f9924a00640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9920160090 0x7f99200cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.952+0000 7f9924a00640 1 -- 172.21.15.131:0/1711279714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9920145450 msgr2=0x7f99200cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.952+0000 7f9924a00640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9920145450 0x7f99200cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.952+0000 7f9924a00640 1 -- 172.21.15.131:0/1711279714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9920143d30 con 0x7f99200b52d0 2024-04-28T12:43:21.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.952+0000 7f9924a00640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99200b52d0 0x7f99201437f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9910000c00 tx=0x7f99100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:21.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.952+0000 7f991d600640 1 -- 172.21.15.131:0/1711279714 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910041070 con 0x7f99200b52d0 2024-04-28T12:43:21.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.952+0000 7f991d600640 1 -- 172.21.15.131:0/1711279714 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991003c870 con 0x7f99200b52d0 2024-04-28T12:43:21.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.952+0000 7f991d600640 1 -- 172.21.15.131:0/1711279714 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910037040 con 0x7f99200b52d0 2024-04-28T12:43:21.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.952+0000 7f9925e00640 1 -- 172.21.15.131:0/1711279714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992015c270 con 0x7f99200b52d0 2024-04-28T12:43:21.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.953+0000 7f9925e00640 1 -- 172.21.15.131:0/1711279714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f992015c830 con 0x7f99200b52d0 2024-04-28T12:43:21.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.953+0000 7f9925e00640 1 -- 172.21.15.131:0/1711279714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9920069350 con 0x7f99200b52d0 2024-04-28T12:43:21.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.954+0000 7f991d600640 1 -- 172.21.15.131:0/1711279714 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9910044070 con 0x7f99200b52d0 2024-04-28T12:43:21.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.955+0000 7f991d600640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f98fc06c8f0 0x7f98fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.955+0000 7f991d600640 1 -- 172.21.15.131:0/1711279714 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f99100ac600 con 0x7f99200b52d0 2024-04-28T12:43:21.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.956+0000 7f991d600640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] conn(0x7f98fc072d90 0x7f98fc0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:21.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.956+0000 7f991d600640 1 -- 172.21.15.131:0/1711279714 --> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f991000af30 con 0x7f98fc072d90 2024-04-28T12:43:21.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.956+0000 7f991d600640 1 -- 172.21.15.131:0/1711279714 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f99100b3050 con 0x7f99200b52d0 2024-04-28T12:43:21.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.956+0000 7f9925400640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] conn(0x7f98fc072d90 0x7f98fc0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.956+0000 7f9917e00640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f98fc06c8f0 0x7f98fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:21.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.956+0000 7f9925400640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] conn(0x7f98fc072d90 0x7f98fc0751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:21.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.957+0000 7f9917e00640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f98fc06c8f0 0x7f98fc06edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9908007fd0 tx=0x7f9908009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:21.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.957+0000 7f991d600640 1 -- 172.21.15.131:0/1711279714 <== osd.3 v2:172.21.15.131:6812/708415318 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f991000af30 con 0x7f98fc072d90 2024-04-28T12:43:21.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.972+0000 7f9925e00640 1 -- 172.21.15.131:0/1711279714 --> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f992015ccc0 con 0x7f98fc072d90 2024-04-28T12:43:21.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.972+0000 7f991d600640 1 -- 172.21.15.131:0/1711279714 <== osd.3 v2:172.21.15.131:6812/708415318 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f992015ccc0 con 0x7f98fc072d90 2024-04-28T12:43:21.968 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:43:21.968 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T12:43:21.968 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:43:21.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.972+0000 7f9925e00640 1 -- 172.21.15.131:0/1711279714 >> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] conn(0x7f98fc072d90 msgr2=0x7f98fc0751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.972+0000 7f9925e00640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.131:6812/708415318,v1:172.21.15.131:6815/708415318] conn(0x7f98fc072d90 0x7f98fc0751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.972+0000 7f9925e00640 1 -- 172.21.15.131:0/1711279714 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f98fc06c8f0 msgr2=0x7f98fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.972+0000 7f9925e00640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f98fc06c8f0 0x7f98fc06edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9908007fd0 tx=0x7f9908009040 comp rx=0 tx=0).stop 2024-04-28T12:43:21.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.972+0000 7f9925e00640 1 -- 172.21.15.131:0/1711279714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99200b52d0 msgr2=0x7f99201437f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:21.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.972+0000 7f9925e00640 1 --2- 172.21.15.131:0/1711279714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99200b52d0 0x7f99201437f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9910000c00 tx=0x7f99100046c0 comp rx=0 tx=0).stop 2024-04-28T12:43:21.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.972+0000 7f9925400640 1 -- 172.21.15.131:0/1711279714 reap_dead start 2024-04-28T12:43:21.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.973+0000 7f9925e00640 1 -- 172.21.15.131:0/1711279714 shutdown_connections 2024-04-28T12:43:21.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.973+0000 7f9925e00640 1 -- 172.21.15.131:0/1711279714 >> 172.21.15.131:0/1711279714 conn(0x7f99200b26b0 msgr2=0x7f99200cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:21.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.973+0000 7f9925e00640 1 -- 172.21.15.131:0/1711279714 shutdown_connections 2024-04-28T12:43:21.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:21.973+0000 7f9925e00640 1 -- 172.21.15.131:0/1711279714 wait complete. 2024-04-28T12:43:21.985 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-28T12:43:22.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.189+0000 7f80cde00640 1 Processor -- start 2024-04-28T12:43:22.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.190+0000 7f80cde00640 1 -- start start 2024-04-28T12:43:22.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.191+0000 7f80cde00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f80c8160420 0x7f80c80d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.191+0000 7f80cde00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f80c8160c20 0x7f80c8161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.191+0000 7f80cde00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80c80cec70 con 0x7f80c81618a0 2024-04-28T12:43:22.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.191+0000 7f80cde00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f80c8070770 con 0x7f80c8160c20 2024-04-28T12:43:22.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.191+0000 7f80cde00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f80c81615d0 con 0x7f80c8160420 2024-04-28T12:43:22.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.191+0000 7f80c7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f80c8160c20 0x7f80c8161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.192+0000 7f80c7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f80c8160c20 0x7f80c8161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48704/0 (socket says 172.21.15.131:48704) 2024-04-28T12:43:22.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.192+0000 7f80c7400640 1 -- 172.21.15.131:0/980068633 learned_addr learned my addr 172.21.15.131:0/980068633 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:22.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.192+0000 7f80c7400640 1 -- 172.21.15.131:0/980068633 >> v2:172.21.15.157:3300/0 conn(0x7f80c8160420 msgr2=0x7f80c80d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:22.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.192+0000 7f80c7400640 1 --2- 172.21.15.131:0/980068633 >> v2:172.21.15.157:3300/0 conn(0x7f80c8160420 0x7f80c80d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.192+0000 7f80c7400640 1 -- 172.21.15.131:0/980068633 >> v1:172.21.15.157:6789/0 conn(0x7f80c81618a0 legacy=0x7f80c8070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:22.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.192+0000 7f80c7400640 1 -- 172.21.15.131:0/980068633 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c80ce3d0 con 0x7f80c8160c20 2024-04-28T12:43:22.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.193+0000 7f80c7400640 1 --2- 172.21.15.131:0/980068633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80c8160c20 0x7f80c8161090 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f80c8072790 tx=0x7f80c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d944f41d16dcd6bc server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.194+0000 7f80c5600640 1 -- 172.21.15.131:0/980068633 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0037070 con 0x7f80c8160c20 2024-04-28T12:43:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.194+0000 7f80c5600640 1 -- 172.21.15.131:0/980068633 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c0032440 con 0x7f80c8160c20 2024-04-28T12:43:22.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.194+0000 7f80c5600640 1 -- 172.21.15.131:0/980068633 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0032810 con 0x7f80c8160c20 2024-04-28T12:43:22.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.194+0000 7f80cde00640 1 -- 172.21.15.131:0/980068633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80c8160c20 msgr2=0x7f80c8161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.194+0000 7f80cde00640 1 --2- 172.21.15.131:0/980068633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80c8160c20 0x7f80c8161090 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f80c8072790 tx=0x7f80c002a270 comp rx=0 tx=0).stop 2024-04-28T12:43:22.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.194+0000 7f80cde00640 1 -- 172.21.15.131:0/980068633 shutdown_connections 2024-04-28T12:43:22.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.194+0000 7f80cde00640 1 --2- 172.21.15.131:0/980068633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80c8160c20 0x7f80c8161090 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.194+0000 7f80cde00640 1 --2- 172.21.15.131:0/980068633 >> v2:172.21.15.157:3300/0 conn(0x7f80c8160420 0x7f80c80d38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.194+0000 7f80cde00640 1 -- 172.21.15.131:0/980068633 >> 172.21.15.131:0/980068633 conn(0x7f80c80d2db0 msgr2=0x7f80c80d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:22.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.194+0000 7f80cde00640 1 -- 172.21.15.131:0/980068633 shutdown_connections 2024-04-28T12:43:22.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.194+0000 7f80cde00640 1 -- 172.21.15.131:0/980068633 wait complete. 2024-04-28T12:43:22.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.196+0000 7f80cde00640 1 Processor -- start 2024-04-28T12:43:22.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.196+0000 7f80cde00640 1 -- start start 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.197+0000 7f80cde00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c8160420 0x7f80c806de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.197+0000 7f80cde00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c8160c20 0x7f80c80c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.197+0000 7f80cde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80c81618a0 0x7f80c80c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.197+0000 7f80cde00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f80c80ce990 con 0x7f80c81618a0 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.197+0000 7f80cde00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f80c80d0d90 con 0x7f80c8160420 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.197+0000 7f80cde00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f80c80d0a20 con 0x7f80c8160c20 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.197+0000 7f80c7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80c81618a0 0x7f80c80c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.197+0000 7f80c7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80c81618a0 0x7f80c80c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48710/0 (socket says 172.21.15.131:48710) 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.197+0000 7f80c7400640 1 -- 172.21.15.131:0/1503238846 learned_addr learned my addr 172.21.15.131:0/1503238846 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.198+0000 7f80c7400640 1 -- 172.21.15.131:0/1503238846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c8160c20 msgr2=0x7f80c80c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.198+0000 7f80c7400640 1 --2- 172.21.15.131:0/1503238846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c8160c20 0x7f80c80c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.198+0000 7f80c7400640 1 -- 172.21.15.131:0/1503238846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c8160420 msgr2=0x7f80c806de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.198+0000 7f80c7400640 1 --2- 172.21.15.131:0/1503238846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c8160420 0x7f80c806de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.198+0000 7f80c7400640 1 -- 172.21.15.131:0/1503238846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c80c96a0 con 0x7f80c81618a0 2024-04-28T12:43:22.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.198+0000 7f80c7400640 1 --2- 172.21.15.131:0/1503238846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80c81618a0 0x7f80c80c8830 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f80c00097f0 tx=0x7f80c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:22.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.198+0000 7f80bf400640 1 -- 172.21.15.131:0/1503238846 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0041070 con 0x7f80c81618a0 2024-04-28T12:43:22.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.198+0000 7f80bf400640 1 -- 172.21.15.131:0/1503238846 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c003c890 con 0x7f80c81618a0 2024-04-28T12:43:22.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.198+0000 7f80bf400640 1 -- 172.21.15.131:0/1503238846 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0037040 con 0x7f80c81618a0 2024-04-28T12:43:22.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.198+0000 7f80cde00640 1 -- 172.21.15.131:0/1503238846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c806e4c0 con 0x7f80c81618a0 2024-04-28T12:43:22.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.199+0000 7f80cde00640 1 -- 172.21.15.131:0/1503238846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c806ea80 con 0x7f80c81618a0 2024-04-28T12:43:22.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.200+0000 7f80cde00640 1 -- 172.21.15.131:0/1503238846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c8069350 con 0x7f80c81618a0 2024-04-28T12:43:22.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.201+0000 7f80bf400640 1 -- 172.21.15.131:0/1503238846 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80c0044070 con 0x7f80c81618a0 2024-04-28T12:43:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.202+0000 7f80bf400640 1 --2- 172.21.15.131:0/1503238846 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f80a406c8f0 0x7f80a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.202+0000 7f80bf400640 1 -- 172.21.15.131:0/1503238846 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f80c00ac700 con 0x7f80c81618a0 2024-04-28T12:43:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.202+0000 7f80c6a00640 1 --2- 172.21.15.131:0/1503238846 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f80a406c8f0 0x7f80a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.203+0000 7f80c6a00640 1 --2- 172.21.15.131:0/1503238846 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f80a406c8f0 0x7f80a406edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009670 tx=0x7f80b8006bf0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:22.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.206+0000 7f80bf400640 1 -- 172.21.15.131:0/1503238846 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f80c80c96a0 con 0x7f80c81618a0 2024-04-28T12:43:22.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.412+0000 7f80cde00640 1 -- 172.21.15.131:0/1503238846 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f80c80cf1c0 con 0x7f80a406c8f0 2024-04-28T12:43:22.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.413+0000 7f80bf400640 1 -- 172.21.15.131:0/1503238846 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f80c80cf1c0 con 0x7f80a406c8f0 2024-04-28T12:43:22.409 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 3 to deep-scrub 2024-04-28T12:43:22.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.415+0000 7f80cde00640 1 -- 172.21.15.131:0/1503238846 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f80a406c8f0 msgr2=0x7f80a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.415+0000 7f80cde00640 1 --2- 172.21.15.131:0/1503238846 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f80a406c8f0 0x7f80a406edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009670 tx=0x7f80b8006bf0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.415+0000 7f80cde00640 1 -- 172.21.15.131:0/1503238846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80c81618a0 msgr2=0x7f80c80c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.415+0000 7f80cde00640 1 --2- 172.21.15.131:0/1503238846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80c81618a0 0x7f80c80c8830 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f80c00097f0 tx=0x7f80c00046c0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.416+0000 7f80cde00640 1 -- 172.21.15.131:0/1503238846 shutdown_connections 2024-04-28T12:43:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.416+0000 7f80cde00640 1 --2- 172.21.15.131:0/1503238846 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f80a406c8f0 0x7f80a406edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.416+0000 7f80cde00640 1 --2- 172.21.15.131:0/1503238846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80c81618a0 0x7f80c80c8830 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.416+0000 7f80cde00640 1 --2- 172.21.15.131:0/1503238846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80c8160c20 0x7f80c80c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.416+0000 7f80cde00640 1 --2- 172.21.15.131:0/1503238846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80c8160420 0x7f80c806de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.416+0000 7f80cde00640 1 -- 172.21.15.131:0/1503238846 >> 172.21.15.131:0/1503238846 conn(0x7f80c80d2db0 msgr2=0x7f80c8070060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.416+0000 7f80cde00640 1 -- 172.21.15.131:0/1503238846 shutdown_connections 2024-04-28T12:43:22.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.416+0000 7f80cde00640 1 -- 172.21.15.131:0/1503238846 wait complete. 2024-04-28T12:43:22.430 INFO:tasks.ceph:Scrubbing osd.4 2024-04-28T12:43:22.430 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-28T12:43:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.637+0000 7f7edae00640 1 Processor -- start 2024-04-28T12:43:22.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.638+0000 7f7edae00640 1 -- start start 2024-04-28T12:43:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.639+0000 7f7edae00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7ed40b52d0 0x7f7ed4071c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.639+0000 7f7edae00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed40738c0 con 0x7f7ed4145450 2024-04-28T12:43:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.639+0000 7f7edae00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed4073e40 con 0x7f7ed406f6c0 2024-04-28T12:43:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.639+0000 7f7edae00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed406fb40 con 0x7f7ed40b52d0 2024-04-28T12:43:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.640+0000 7f7ed9a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7ed4145450 0x7f7ed4160090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39610/0 (socket says 172.21.15.131:39610) 2024-04-28T12:43:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.640+0000 7f7ed9a00640 1 -- 172.21.15.131:0/221923225 learned_addr learned my addr 172.21.15.131:0/221923225 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:22.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.640+0000 7f7eda400640 1 --2- 172.21.15.131:0/221923225 >> v2:172.21.15.131:3300/0 conn(0x7f7ed40b52d0 0x7f7ed4071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.640+0000 7f7ed3e00640 1 -- 172.21.15.131:0/221923225 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2757259823 0 0) 0x7f7ed40738c0 con 0x7f7ed4145450 2024-04-28T12:43:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.640+0000 7f7ed3e00640 1 -- 172.21.15.131:0/221923225 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ebc005730 con 0x7f7ed4145450 2024-04-28T12:43:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.641+0000 7f7eda400640 1 -- 172.21.15.131:0/221923225 >> v1:172.21.15.157:6789/0 conn(0x7f7ed406f6c0 legacy=0x7f7ed40b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.641+0000 7f7eda400640 1 -- 172.21.15.131:0/221923225 >> v1:172.21.15.131:6789/0 conn(0x7f7ed4145450 legacy=0x7f7ed4160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.641+0000 7f7eda400640 1 -- 172.21.15.131:0/221923225 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed4073b60 con 0x7f7ed40b52d0 2024-04-28T12:43:22.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.642+0000 7f7eda400640 1 --2- 172.21.15.131:0/221923225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ed40b52d0 0x7f7ed4071c00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009d90 tx=0x7f7ecc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3232ed00945be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:22.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.642+0000 7f7ed3e00640 1 -- 172.21.15.131:0/221923225 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc007740 con 0x7f7ed40b52d0 2024-04-28T12:43:22.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.642+0000 7f7ed3e00640 1 -- 172.21.15.131:0/221923225 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ecc0078e0 con 0x7f7ed40b52d0 2024-04-28T12:43:22.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.642+0000 7f7ed3e00640 1 -- 172.21.15.131:0/221923225 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc007cb0 con 0x7f7ed40b52d0 2024-04-28T12:43:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.643+0000 7f7edae00640 1 -- 172.21.15.131:0/221923225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ed40b52d0 msgr2=0x7f7ed4071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.643+0000 7f7edae00640 1 --2- 172.21.15.131:0/221923225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ed40b52d0 0x7f7ed4071c00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009d90 tx=0x7f7ecc02a6f0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.643+0000 7f7edae00640 1 -- 172.21.15.131:0/221923225 shutdown_connections 2024-04-28T12:43:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.643+0000 7f7edae00640 1 --2- 172.21.15.131:0/221923225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ed40b52d0 0x7f7ed4071c00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.643+0000 7f7edae00640 1 -- 172.21.15.131:0/221923225 >> 172.21.15.131:0/221923225 conn(0x7f7ed40b26b0 msgr2=0x7f7ed40b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.643+0000 7f7edae00640 1 -- 172.21.15.131:0/221923225 shutdown_connections 2024-04-28T12:43:22.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.643+0000 7f7edae00640 1 -- 172.21.15.131:0/221923225 wait complete. 2024-04-28T12:43:22.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.644+0000 7f7edae00640 1 Processor -- start 2024-04-28T12:43:22.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.645+0000 7f7edae00640 1 -- start start 2024-04-28T12:43:22.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.645+0000 7f7edae00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ed406f6c0 0x7f7ed41f1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7edae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed40b52d0 0x7f7ed413c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7edae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed4145450 0x7f7ed40b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7edae00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed40cd4e0 con 0x7f7ed406f6c0 2024-04-28T12:43:22.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7edae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed406eec0 con 0x7f7ed4145450 2024-04-28T12:43:22.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7edae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed40cd660 con 0x7f7ed40b52d0 2024-04-28T12:43:22.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7ed9a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ed406f6c0 0x7f7ed41f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7ed9a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ed406f6c0 0x7f7ed41f1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48732/0 (socket says 172.21.15.131:48732) 2024-04-28T12:43:22.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7ed9a00640 1 -- 172.21.15.131:0/896521194 learned_addr learned my addr 172.21.15.131:0/896521194 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:22.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7eda400640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed4145450 0x7f7ed40b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7ed9000640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed40b52d0 0x7f7ed413c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7ed9a00640 1 -- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed40b52d0 msgr2=0x7f7ed413c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7ed9a00640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed40b52d0 0x7f7ed413c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7ed9a00640 1 -- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed4145450 msgr2=0x7f7ed40b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7ed9a00640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed4145450 0x7f7ed40b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7ed9a00640 1 -- 172.21.15.131:0/896521194 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed413ce80 con 0x7f7ed406f6c0 2024-04-28T12:43:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.646+0000 7f7eda400640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed4145450 0x7f7ed40b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.647+0000 7f7ed9a00640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ed406f6c0 0x7f7ed41f1000 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800c500 tx=0x7f7ec800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.647+0000 7f7ed1600640 1 -- 172.21.15.131:0/896521194 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8017070 con 0x7f7ed406f6c0 2024-04-28T12:43:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.647+0000 7f7ed1600640 1 -- 172.21.15.131:0/896521194 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec8005ce0 con 0x7f7ed406f6c0 2024-04-28T12:43:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.647+0000 7f7ed1600640 1 -- 172.21.15.131:0/896521194 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8007330 con 0x7f7ed406f6c0 2024-04-28T12:43:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.647+0000 7f7edae00640 1 -- 172.21.15.131:0/896521194 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed413d160 con 0x7f7ed406f6c0 2024-04-28T12:43:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.647+0000 7f7edae00640 1 -- 172.21.15.131:0/896521194 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed41f0800 con 0x7f7ed406f6c0 2024-04-28T12:43:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.648+0000 7f7edae00640 1 -- 172.21.15.131:0/896521194 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ed4069350 con 0x7f7ed406f6c0 2024-04-28T12:43:22.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.649+0000 7f7ed1600640 1 -- 172.21.15.131:0/896521194 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ec801a070 con 0x7f7ed406f6c0 2024-04-28T12:43:22.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.650+0000 7f7ed1600640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7eb406c8f0 0x7f7eb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.650+0000 7f7ed1600640 1 -- 172.21.15.131:0/896521194 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f7ec808b510 con 0x7f7ed406f6c0 2024-04-28T12:43:22.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.650+0000 7f7ed1600640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] conn(0x7f7eb4072d90 0x7f7eb40751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.650+0000 7f7ed1600640 1 -- 172.21.15.131:0/896521194 --> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7eb40758a0 con 0x7f7eb4072d90 2024-04-28T12:43:22.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.650+0000 7f7ed1600640 1 -- 172.21.15.131:0/896521194 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7ec808b9a0 con 0x7f7ed406f6c0 2024-04-28T12:43:22.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.651+0000 7f7ed9000640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7eb406c8f0 0x7f7eb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.651+0000 7f7eda400640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] conn(0x7f7eb4072d90 0x7f7eb40751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.652+0000 7f7ed9000640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7eb406c8f0 0x7f7eb406edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40cfb20 tx=0x7f7ec0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.652+0000 7f7eda400640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] conn(0x7f7eb4072d90 0x7f7eb40751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:22.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.655+0000 7f7ed1600640 1 -- 172.21.15.131:0/896521194 <== osd.4 v2:172.21.15.157:6802/4053377461 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7eb40758a0 con 0x7f7eb4072d90 2024-04-28T12:43:22.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.669+0000 7f7edae00640 1 -- 172.21.15.131:0/896521194 --> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7ed40b3310 con 0x7f7eb4072d90 2024-04-28T12:43:22.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.670+0000 7f7ed1600640 1 -- 172.21.15.131:0/896521194 <== osd.4 v2:172.21.15.157:6802/4053377461 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7ed40b3310 con 0x7f7eb4072d90 2024-04-28T12:43:22.667 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:43:22.667 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T12:43:22.667 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:43:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.670+0000 7f7edae00640 1 -- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] conn(0x7f7eb4072d90 msgr2=0x7f7eb40751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.670+0000 7f7edae00640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:6802/4053377461,v1:172.21.15.157:6803/4053377461] conn(0x7f7eb4072d90 0x7f7eb40751d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.671+0000 7f7edae00640 1 -- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7eb406c8f0 msgr2=0x7f7eb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.671+0000 7f7edae00640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f7eb406c8f0 0x7f7eb406edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40cfb20 tx=0x7f7ec0009040 comp rx=0 tx=0).stop 2024-04-28T12:43:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.671+0000 7f7edae00640 1 -- 172.21.15.131:0/896521194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ed406f6c0 msgr2=0x7f7ed41f1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.671+0000 7f7edae00640 1 --2- 172.21.15.131:0/896521194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ed406f6c0 0x7f7ed41f1000 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800c500 tx=0x7f7ec800c9c0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.671+0000 7f7eda400640 1 -- 172.21.15.131:0/896521194 reap_dead start 2024-04-28T12:43:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.671+0000 7f7edae00640 1 -- 172.21.15.131:0/896521194 shutdown_connections 2024-04-28T12:43:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.671+0000 7f7edae00640 1 -- 172.21.15.131:0/896521194 >> 172.21.15.131:0/896521194 conn(0x7f7ed40b26b0 msgr2=0x7f7ed40b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.671+0000 7f7edae00640 1 -- 172.21.15.131:0/896521194 shutdown_connections 2024-04-28T12:43:22.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.671+0000 7f7edae00640 1 -- 172.21.15.131:0/896521194 wait complete. 2024-04-28T12:43:22.685 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-28T12:43:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.888+0000 7faf53600640 1 Processor -- start 2024-04-28T12:43:22.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.888+0000 7faf53600640 1 -- start start 2024-04-28T12:43:22.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.890+0000 7faf53600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf4c1467c0 0x7faf4c072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.890+0000 7faf53600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7faf4c0b3ce0 0x7faf4c0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.890+0000 7faf53600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf4c146ba0 con 0x7faf4c0b4140 2024-04-28T12:43:22.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.890+0000 7faf53600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7faf4c073dd0 con 0x7faf4c0b3ce0 2024-04-28T12:43:22.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.890+0000 7faf53600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faf4c0731c0 con 0x7faf4c1467c0 2024-04-28T12:43:22.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.891+0000 7faf50c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf4c1467c0 0x7faf4c072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.891+0000 7faf50c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf4c1467c0 0x7faf4c072b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:45506/0 (socket says 172.21.15.131:45506) 2024-04-28T12:43:22.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.891+0000 7faf50c00640 1 -- 172.21.15.131:0/1785232169 learned_addr learned my addr 172.21.15.131:0/1785232169 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:22.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.891+0000 7faf4be00640 1 --2- 172.21.15.131:0/1785232169 >> v2:172.21.15.157:3300/0 conn(0x7faf4c0b3ce0 0x7faf4c0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.891+0000 7faf50c00640 1 -- 172.21.15.131:0/1785232169 >> v2:172.21.15.157:3300/0 conn(0x7faf4c0b3ce0 msgr2=0x7faf4c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:22.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.891+0000 7faf50c00640 1 --2- 172.21.15.131:0/1785232169 >> v2:172.21.15.157:3300/0 conn(0x7faf4c0b3ce0 0x7faf4c0d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.891+0000 7faf50c00640 1 -- 172.21.15.131:0/1785232169 >> v1:172.21.15.157:6789/0 conn(0x7faf4c0b4140 legacy=0x7faf4c160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.891+0000 7faf50c00640 1 -- 172.21.15.131:0/1785232169 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c0734a0 con 0x7faf4c1467c0 2024-04-28T12:43:22.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.893+0000 7faf50c00640 1 --2- 172.21.15.131:0/1785232169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf4c1467c0 0x7faf4c072b70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf4c073f50 tx=0x7faf4002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131c5d1089d2ad86 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.893+0000 7faf4b400640 1 -- 172.21.15.131:0/1785232169 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40037070 con 0x7faf4c1467c0 2024-04-28T12:43:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.894+0000 7faf4b400640 1 -- 172.21.15.131:0/1785232169 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf40032440 con 0x7faf4c1467c0 2024-04-28T12:43:22.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.894+0000 7faf4b400640 1 -- 172.21.15.131:0/1785232169 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40032810 con 0x7faf4c1467c0 2024-04-28T12:43:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.894+0000 7faf53600640 1 -- 172.21.15.131:0/1785232169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf4c1467c0 msgr2=0x7faf4c072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.894+0000 7faf53600640 1 --2- 172.21.15.131:0/1785232169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf4c1467c0 0x7faf4c072b70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf4c073f50 tx=0x7faf4002a250 comp rx=0 tx=0).stop 2024-04-28T12:43:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.894+0000 7faf53600640 1 -- 172.21.15.131:0/1785232169 shutdown_connections 2024-04-28T12:43:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.894+0000 7faf53600640 1 --2- 172.21.15.131:0/1785232169 >> v2:172.21.15.157:3300/0 conn(0x7faf4c0b3ce0 0x7faf4c0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.894+0000 7faf53600640 1 --2- 172.21.15.131:0/1785232169 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf4c1467c0 0x7faf4c072b70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.894+0000 7faf53600640 1 -- 172.21.15.131:0/1785232169 >> 172.21.15.131:0/1785232169 conn(0x7faf4c0b2900 msgr2=0x7faf4c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.894+0000 7faf53600640 1 -- 172.21.15.131:0/1785232169 shutdown_connections 2024-04-28T12:43:22.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.894+0000 7faf53600640 1 -- 172.21.15.131:0/1785232169 wait complete. 2024-04-28T12:43:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.896+0000 7faf53600640 1 Processor -- start 2024-04-28T12:43:22.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.896+0000 7faf53600640 1 -- start start 2024-04-28T12:43:22.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.897+0000 7faf53600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf4c0b3ce0 0x7faf4c1e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.897+0000 7faf53600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf4c1467c0 0x7faf4c0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.897+0000 7faf53600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4c0cfcb0 0x7faf4c1438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.897+0000 7faf53600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c06f000 con 0x7faf4c0cfcb0 2024-04-28T12:43:22.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.897+0000 7faf53600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c073a60 con 0x7faf4c0b3ce0 2024-04-28T12:43:22.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.897+0000 7faf53600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faf4c06f180 con 0x7faf4c1467c0 2024-04-28T12:43:22.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.897+0000 7faf51600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4c0cfcb0 0x7faf4c1438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.898+0000 7faf51600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4c0cfcb0 0x7faf4c1438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48736/0 (socket says 172.21.15.131:48736) 2024-04-28T12:43:22.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.898+0000 7faf51600640 1 -- 172.21.15.131:0/917649456 learned_addr learned my addr 172.21.15.131:0/917649456 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:22.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.898+0000 7faf50c00640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf4c0b3ce0 0x7faf4c1e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.898+0000 7faf51600640 1 -- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf4c1467c0 msgr2=0x7faf4c0cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.898+0000 7faf51600640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf4c1467c0 0x7faf4c0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.898+0000 7faf51600640 1 -- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf4c0b3ce0 msgr2=0x7faf4c1e65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:22.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.898+0000 7faf51600640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf4c0b3ce0 0x7faf4c1e65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:22.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.898+0000 7faf51600640 1 -- 172.21.15.131:0/917649456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c144640 con 0x7faf4c0cfcb0 2024-04-28T12:43:22.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.898+0000 7faf50c00640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf4c0b3ce0 0x7faf4c1e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:22.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.898+0000 7faf51600640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4c0cfcb0 0x7faf4c1438a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7faf4400c730 tx=0x7faf4400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:22.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.899+0000 7faf48c00640 1 -- 172.21.15.131:0/917649456 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44003b70 con 0x7faf4c0cfcb0 2024-04-28T12:43:22.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.899+0000 7faf48c00640 1 -- 172.21.15.131:0/917649456 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf44003d10 con 0x7faf4c0cfcb0 2024-04-28T12:43:22.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.899+0000 7faf48c00640 1 -- 172.21.15.131:0/917649456 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4400aa90 con 0x7faf4c0cfcb0 2024-04-28T12:43:22.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.899+0000 7faf53600640 1 -- 172.21.15.131:0/917649456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4c144920 con 0x7faf4c0cfcb0 2024-04-28T12:43:22.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.899+0000 7faf53600640 1 -- 172.21.15.131:0/917649456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4c144ee0 con 0x7faf4c0cfcb0 2024-04-28T12:43:22.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.900+0000 7faf53600640 1 -- 172.21.15.131:0/917649456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf4c069350 con 0x7faf4c0cfcb0 2024-04-28T12:43:22.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.901+0000 7faf48c00640 1 -- 172.21.15.131:0/917649456 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf4400ac30 con 0x7faf4c0cfcb0 2024-04-28T12:43:22.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.902+0000 7faf48c00640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faf2c06c8f0 0x7faf2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:22.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.902+0000 7faf48c00640 1 -- 172.21.15.131:0/917649456 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7faf4408b950 con 0x7faf4c0cfcb0 2024-04-28T12:43:22.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.903+0000 7faf50c00640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faf2c06c8f0 0x7faf2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:22.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.904+0000 7faf50c00640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faf2c06c8f0 0x7faf2c06edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faf40034000 tx=0x7faf4002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:22.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:22.907+0000 7faf48c00640 1 -- 172.21.15.131:0/917649456 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf44059e50 con 0x7faf4c0cfcb0 2024-04-28T12:43:23.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.108+0000 7faf53600640 1 -- 172.21.15.131:0/917649456 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7faf4c0aef90 con 0x7faf2c06c8f0 2024-04-28T12:43:23.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.109+0000 7faf48c00640 1 -- 172.21.15.131:0/917649456 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7faf4c0aef90 con 0x7faf2c06c8f0 2024-04-28T12:43:23.105 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 4 to deep-scrub 2024-04-28T12:43:23.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.111+0000 7faf53600640 1 -- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faf2c06c8f0 msgr2=0x7faf2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.111+0000 7faf53600640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faf2c06c8f0 0x7faf2c06edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faf40034000 tx=0x7faf4002e000 comp rx=0 tx=0).stop 2024-04-28T12:43:23.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.111+0000 7faf53600640 1 -- 172.21.15.131:0/917649456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4c0cfcb0 msgr2=0x7faf4c1438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.111+0000 7faf53600640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4c0cfcb0 0x7faf4c1438a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7faf4400c730 tx=0x7faf4400cbf0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.112+0000 7faf53600640 1 -- 172.21.15.131:0/917649456 shutdown_connections 2024-04-28T12:43:23.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.112+0000 7faf53600640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7faf2c06c8f0 0x7faf2c06edb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.112+0000 7faf53600640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf4c0cfcb0 0x7faf4c1438a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.112+0000 7faf53600640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf4c1467c0 0x7faf4c0cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.112+0000 7faf53600640 1 --2- 172.21.15.131:0/917649456 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf4c0b3ce0 0x7faf4c1e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.112+0000 7faf53600640 1 -- 172.21.15.131:0/917649456 >> 172.21.15.131:0/917649456 conn(0x7faf4c0b2900 msgr2=0x7faf4c0d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:23.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.112+0000 7faf53600640 1 -- 172.21.15.131:0/917649456 shutdown_connections 2024-04-28T12:43:23.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.112+0000 7faf53600640 1 -- 172.21.15.131:0/917649456 wait complete. 2024-04-28T12:43:23.127 INFO:tasks.ceph:Scrubbing osd.5 2024-04-28T12:43:23.127 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-28T12:43:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.336+0000 7fa60be00640 1 Processor -- start 2024-04-28T12:43:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.337+0000 7fa60be00640 1 -- start start 2024-04-28T12:43:23.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.338+0000 7fa60be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa60c145450 0x7fa60c0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:23.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.338+0000 7fa60be00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa60c163690 con 0x7fa60c06f830 2024-04-28T12:43:23.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.338+0000 7fa60be00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa60c06ed50 con 0x7fa60c163070 2024-04-28T12:43:23.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.338+0000 7fa60be00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa60c06f030 con 0x7fa60c145450 2024-04-28T12:43:23.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.339+0000 7fa60aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa60c145450 0x7fa60c0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:23.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.339+0000 7fa60aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa60c145450 0x7fa60c0b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48738/0 (socket says 172.21.15.131:48738) 2024-04-28T12:43:23.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.339+0000 7fa60aa00640 1 -- 172.21.15.131:0/3726752639 learned_addr learned my addr 172.21.15.131:0/3726752639 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:23.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.339+0000 7fa60aa00640 1 -- 172.21.15.131:0/3726752639 >> v1:172.21.15.157:6789/0 conn(0x7fa60c163070 legacy=0x7fa60c072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.339+0000 7fa60aa00640 1 -- 172.21.15.131:0/3726752639 >> v1:172.21.15.131:6789/0 conn(0x7fa60c06f830 legacy=0x7fa60c162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.339+0000 7fa60aa00640 1 -- 172.21.15.131:0/3726752639 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60c0b5210 con 0x7fa60c145450 2024-04-28T12:43:23.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.340+0000 7fa60aa00640 1 --2- 172.21.15.131:0/3726752639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa60c145450 0x7fa60c0b4cd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009870 tx=0x7fa5fc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbf454179bae999 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:23.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.341+0000 7fa60a000640 1 -- 172.21.15.131:0/3726752639 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc037070 con 0x7fa60c145450 2024-04-28T12:43:23.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.341+0000 7fa60a000640 1 -- 172.21.15.131:0/3726752639 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5fc02ae50 con 0x7fa60c145450 2024-04-28T12:43:23.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.341+0000 7fa60a000640 1 -- 172.21.15.131:0/3726752639 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc032860 con 0x7fa60c145450 2024-04-28T12:43:23.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.341+0000 7fa60be00640 1 -- 172.21.15.131:0/3726752639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa60c145450 msgr2=0x7fa60c0b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.341+0000 7fa60be00640 1 --2- 172.21.15.131:0/3726752639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa60c145450 0x7fa60c0b4cd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009870 tx=0x7fa5fc02a300 comp rx=0 tx=0).stop 2024-04-28T12:43:23.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.342+0000 7fa60be00640 1 -- 172.21.15.131:0/3726752639 shutdown_connections 2024-04-28T12:43:23.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.342+0000 7fa60be00640 1 --2- 172.21.15.131:0/3726752639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa60c145450 0x7fa60c0b4cd0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.342+0000 7fa60be00640 1 -- 172.21.15.131:0/3726752639 >> 172.21.15.131:0/3726752639 conn(0x7fa60c0b26b0 msgr2=0x7fa60c0b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:23.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.342+0000 7fa60be00640 1 -- 172.21.15.131:0/3726752639 shutdown_connections 2024-04-28T12:43:23.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.342+0000 7fa60be00640 1 -- 172.21.15.131:0/3726752639 wait complete. 2024-04-28T12:43:23.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.344+0000 7fa60be00640 1 Processor -- start 2024-04-28T12:43:23.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.344+0000 7fa60be00640 1 -- start start 2024-04-28T12:43:23.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.345+0000 7fa60be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60c06f830 0x7fa60c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:23.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.345+0000 7fa60be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa60c145450 0x7fa60c143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:23.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.345+0000 7fa60be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60c163070 0x7fa60c144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:23.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.345+0000 7fa60be00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c0734d0 con 0x7fa60c145450 2024-04-28T12:43:23.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.345+0000 7fa60be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c073350 con 0x7fa60c163070 2024-04-28T12:43:23.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.345+0000 7fa60be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa60c073650 con 0x7fa60c06f830 2024-04-28T12:43:23.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.345+0000 7fa603e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa60c145450 0x7fa60c143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:23.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.346+0000 7fa603e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa60c145450 0x7fa60c143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48742/0 (socket says 172.21.15.131:48742) 2024-04-28T12:43:23.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.346+0000 7fa603e00640 1 -- 172.21.15.131:0/2718648564 learned_addr learned my addr 172.21.15.131:0/2718648564 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:23.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.346+0000 7fa60aa00640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60c06f830 0x7fa60c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:23.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.346+0000 7fa60b400640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60c163070 0x7fa60c144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:23.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.346+0000 7fa603e00640 1 -- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60c06f830 msgr2=0x7fa60c15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.346+0000 7fa603e00640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa60c06f830 0x7fa60c15e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.346+0000 7fa603e00640 1 -- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60c163070 msgr2=0x7fa60c144430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.346+0000 7fa603e00640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60c163070 0x7fa60c144430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.346+0000 7fa603e00640 1 -- 172.21.15.131:0/2718648564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60c0c6a10 con 0x7fa60c145450 2024-04-28T12:43:23.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.346+0000 7fa60b400640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa60c163070 0x7fa60c144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:23.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.347+0000 7fa603e00640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa60c145450 0x7fa60c143ef0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400c400 tx=0x7fa5f400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:23.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.347+0000 7fa602a00640 1 -- 172.21.15.131:0/2718648564 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4017070 con 0x7fa60c145450 2024-04-28T12:43:23.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.347+0000 7fa602a00640 1 -- 172.21.15.131:0/2718648564 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f40052d0 con 0x7fa60c145450 2024-04-28T12:43:23.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.347+0000 7fa602a00640 1 -- 172.21.15.131:0/2718648564 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4003830 con 0x7fa60c145450 2024-04-28T12:43:23.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.347+0000 7fa60be00640 1 -- 172.21.15.131:0/2718648564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60c0c6c80 con 0x7fa60c145450 2024-04-28T12:43:23.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.347+0000 7fa60be00640 1 -- 172.21.15.131:0/2718648564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60c0c7240 con 0x7fa60c145450 2024-04-28T12:43:23.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.348+0000 7fa60be00640 1 -- 172.21.15.131:0/2718648564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa60c069350 con 0x7fa60c145450 2024-04-28T12:43:23.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.349+0000 7fa602a00640 1 -- 172.21.15.131:0/2718648564 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5f401a070 con 0x7fa60c145450 2024-04-28T12:43:23.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.350+0000 7fa602a00640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa5e806c8f0 0x7fa5e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:23.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.350+0000 7fa602a00640 1 -- 172.21.15.131:0/2718648564 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fa5f408ae70 con 0x7fa60c145450 2024-04-28T12:43:23.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.350+0000 7fa602a00640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] conn(0x7fa5e8072d90 0x7fa5e80751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:23.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.350+0000 7fa602a00640 1 -- 172.21.15.131:0/2718648564 --> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5e80758a0 con 0x7fa5e8072d90 2024-04-28T12:43:23.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.350+0000 7fa602a00640 1 -- 172.21.15.131:0/2718648564 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa5f408d490 con 0x7fa60c145450 2024-04-28T12:43:23.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.351+0000 7fa60aa00640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa5e806c8f0 0x7fa5e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:23.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.351+0000 7fa60b400640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] conn(0x7fa5e8072d90 0x7fa5e80751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:23.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.352+0000 7fa60b400640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] conn(0x7fa5e8072d90 0x7fa5e80751d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:23.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.352+0000 7fa60aa00640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa5e806c8f0 0x7fa5e806edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc034000 tx=0x7fa5fc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:23.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.354+0000 7fa602a00640 1 -- 172.21.15.131:0/2718648564 <== osd.5 v2:172.21.15.157:6818/2721118630 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa5e80758a0 con 0x7fa5e8072d90 2024-04-28T12:43:23.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.369+0000 7fa60be00640 1 -- 172.21.15.131:0/2718648564 --> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa60c06fec0 con 0x7fa5e8072d90 2024-04-28T12:43:23.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.370+0000 7fa602a00640 1 -- 172.21.15.131:0/2718648564 <== osd.5 v2:172.21.15.157:6818/2721118630 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa60c06fec0 con 0x7fa5e8072d90 2024-04-28T12:43:23.366 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:43:23.366 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T12:43:23.366 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:43:23.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.370+0000 7fa60be00640 1 -- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] conn(0x7fa5e8072d90 msgr2=0x7fa5e80751d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.370+0000 7fa60be00640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:6818/2721118630,v1:172.21.15.157:6819/2721118630] conn(0x7fa5e8072d90 0x7fa5e80751d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.370+0000 7fa60be00640 1 -- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa5e806c8f0 msgr2=0x7fa5e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.370+0000 7fa60be00640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fa5e806c8f0 0x7fa5e806edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc034000 tx=0x7fa5fc02e000 comp rx=0 tx=0).stop 2024-04-28T12:43:23.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.370+0000 7fa60be00640 1 -- 172.21.15.131:0/2718648564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa60c145450 msgr2=0x7fa60c143ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.370+0000 7fa60be00640 1 --2- 172.21.15.131:0/2718648564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa60c145450 0x7fa60c143ef0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400c400 tx=0x7fa5f400c8c0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.370+0000 7fa60b400640 1 -- 172.21.15.131:0/2718648564 reap_dead start 2024-04-28T12:43:23.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.370+0000 7fa60be00640 1 -- 172.21.15.131:0/2718648564 shutdown_connections 2024-04-28T12:43:23.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.370+0000 7fa60be00640 1 -- 172.21.15.131:0/2718648564 >> 172.21.15.131:0/2718648564 conn(0x7fa60c0b26b0 msgr2=0x7fa60c142520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:23.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.371+0000 7fa60be00640 1 -- 172.21.15.131:0/2718648564 shutdown_connections 2024-04-28T12:43:23.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.371+0000 7fa60be00640 1 -- 172.21.15.131:0/2718648564 wait complete. 2024-04-28T12:43:23.385 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-28T12:43:23.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.584+0000 7f178f800640 1 Processor -- start 2024-04-28T12:43:23.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.584+0000 7f178f800640 1 -- start start 2024-04-28T12:43:23.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.586+0000 7f178f800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f178815cab0 0x7f17880afae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.586+0000 7f178f800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f17880d1250 0x7f178815f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.586+0000 7f178f800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17880b0650 con 0x7f17880b0020 2024-04-28T12:43:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.586+0000 7f178f800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f17880d0900 con 0x7f17880d1250 2024-04-28T12:43:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.586+0000 7f178f800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f17881635a0 con 0x7f178815cab0 2024-04-28T12:43:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.586+0000 7f178d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f17880d1250 0x7f178815f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.586+0000 7f178d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f17880d1250 0x7f178815f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48746/0 (socket says 172.21.15.131:48746) 2024-04-28T12:43:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.586+0000 7f178d800640 1 -- 172.21.15.131:0/2900413939 learned_addr learned my addr 172.21.15.131:0/2900413939 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:23.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.587+0000 7f178ce00640 1 --2- 172.21.15.131:0/2900413939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f178815cab0 0x7f17880afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:23.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.587+0000 7f178d800640 1 -- 172.21.15.131:0/2900413939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f178815cab0 msgr2=0x7f17880afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:23.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.587+0000 7f178d800640 1 --2- 172.21.15.131:0/2900413939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f178815cab0 0x7f17880afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.587+0000 7f178d800640 1 -- 172.21.15.131:0/2900413939 >> v1:172.21.15.131:6789/0 conn(0x7f17880b0020 legacy=0x7f17880d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.587+0000 7f178d800640 1 -- 172.21.15.131:0/2900413939 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1788163a30 con 0x7f17880d1250 2024-04-28T12:43:23.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.587+0000 7f178ce00640 1 --2- 172.21.15.131:0/2900413939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f178815cab0 0x7f17880afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:23.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.588+0000 7f178d800640 1 --2- 172.21.15.131:0/2900413939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17880d1250 0x7f178815f5c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1780009e50 tx=0x7f178002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43edba304935051 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:23.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.588+0000 7f1787e00640 1 -- 172.21.15.131:0/2900413939 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780036070 con 0x7f17880d1250 2024-04-28T12:43:23.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.588+0000 7f1787e00640 1 -- 172.21.15.131:0/2900413939 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1780031600 con 0x7f17880d1250 2024-04-28T12:43:23.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.588+0000 7f1787e00640 1 -- 172.21.15.131:0/2900413939 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800319d0 con 0x7f17880d1250 2024-04-28T12:43:23.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.589+0000 7f178f800640 1 -- 172.21.15.131:0/2900413939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17880d1250 msgr2=0x7f178815f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.589+0000 7f178f800640 1 --2- 172.21.15.131:0/2900413939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17880d1250 0x7f178815f5c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1780009e50 tx=0x7f178002a710 comp rx=0 tx=0).stop 2024-04-28T12:43:23.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.589+0000 7f178f800640 1 -- 172.21.15.131:0/2900413939 shutdown_connections 2024-04-28T12:43:23.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.589+0000 7f178f800640 1 --2- 172.21.15.131:0/2900413939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17880d1250 0x7f178815f5c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.589+0000 7f178f800640 1 --2- 172.21.15.131:0/2900413939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f178815cab0 0x7f17880afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.589+0000 7f178f800640 1 -- 172.21.15.131:0/2900413939 >> 172.21.15.131:0/2900413939 conn(0x7f17880b2900 msgr2=0x7f17880b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:23.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.589+0000 7f178f800640 1 -- 172.21.15.131:0/2900413939 shutdown_connections 2024-04-28T12:43:23.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.589+0000 7f178f800640 1 -- 172.21.15.131:0/2900413939 wait complete. 2024-04-28T12:43:23.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.591+0000 7f178f800640 1 Processor -- start 2024-04-28T12:43:23.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.591+0000 7f178f800640 1 -- start start 2024-04-28T12:43:23.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.592+0000 7f178f800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17880b0020 0x7f178814d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:23.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.592+0000 7f178f800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17880d1250 0x7f178815fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:23.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.592+0000 7f178f800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f178815cab0 0x7f1788155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:23.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.592+0000 7f178f800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1788163230 con 0x7f17880d1250 2024-04-28T12:43:23.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.592+0000 7f178f800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1788162f50 con 0x7f17880b0020 2024-04-28T12:43:23.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.592+0000 7f178f800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f178815ce90 con 0x7f178815cab0 2024-04-28T12:43:23.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.592+0000 7f177fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17880d1250 0x7f178815fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:23.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.592+0000 7f177fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17880d1250 0x7f178815fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48752/0 (socket says 172.21.15.131:48752) 2024-04-28T12:43:23.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.592+0000 7f177fe00640 1 -- 172.21.15.131:0/1787174271 learned_addr learned my addr 172.21.15.131:0/1787174271 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:23.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.592+0000 7f178ce00640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17880b0020 0x7f178814d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:23.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.592+0000 7f177fe00640 1 -- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f178815cab0 msgr2=0x7f1788155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.592+0000 7f178d800640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f178815cab0 0x7f1788155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:23.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.593+0000 7f177fe00640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f178815cab0 0x7f1788155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.593+0000 7f177fe00640 1 -- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17880b0020 msgr2=0x7f178814d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.593+0000 7f177fe00640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17880b0020 0x7f178814d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.593+0000 7f177fe00640 1 -- 172.21.15.131:0/1787174271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178814e3c0 con 0x7f17880d1250 2024-04-28T12:43:23.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.593+0000 7f178ce00640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17880b0020 0x7f178814d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:23.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.593+0000 7f177fe00640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17880d1250 0x7f178815fe40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f177000c400 tx=0x7f177000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:23.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.593+0000 7f1785600640 1 -- 172.21.15.131:0/1787174271 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17700052d0 con 0x7f17880d1250 2024-04-28T12:43:23.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.593+0000 7f1785600640 1 -- 172.21.15.131:0/1787174271 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1770005490 con 0x7f17880d1250 2024-04-28T12:43:23.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.593+0000 7f1785600640 1 -- 172.21.15.131:0/1787174271 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770007260 con 0x7f17880d1250 2024-04-28T12:43:23.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.593+0000 7f178f800640 1 -- 172.21.15.131:0/1787174271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1788160890 con 0x7f17880d1250 2024-04-28T12:43:23.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.593+0000 7f178f800640 1 -- 172.21.15.131:0/1787174271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1788160e50 con 0x7f17880d1250 2024-04-28T12:43:23.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.594+0000 7f178f800640 1 -- 172.21.15.131:0/1787174271 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1788069350 con 0x7f17880d1250 2024-04-28T12:43:23.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.595+0000 7f1785600640 1 -- 172.21.15.131:0/1787174271 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1770007400 con 0x7f17880d1250 2024-04-28T12:43:23.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.596+0000 7f1785600640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f176406c8f0 0x7f176406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:23.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.596+0000 7f1785600640 1 -- 172.21.15.131:0/1787174271 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f177008afa0 con 0x7f17880d1250 2024-04-28T12:43:23.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.597+0000 7f178ce00640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f176406c8f0 0x7f176406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:23.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.598+0000 7f178ce00640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f176406c8f0 0x7f176406edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1778006fd0 tx=0x7f1778008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:23.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.601+0000 7f1785600640 1 -- 172.21.15.131:0/1787174271 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17700594b0 con 0x7f17880d1250 2024-04-28T12:43:23.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.802+0000 7f178f800640 1 -- 172.21.15.131:0/1787174271 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f178814d9b0 con 0x7f176406c8f0 2024-04-28T12:43:23.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.803+0000 7f1785600640 1 -- 172.21.15.131:0/1787174271 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1778008580 con 0x7f176406c8f0 2024-04-28T12:43:23.799 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 5 to deep-scrub 2024-04-28T12:43:23.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.805+0000 7f178f800640 1 -- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f176406c8f0 msgr2=0x7f176406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.805+0000 7f178f800640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f176406c8f0 0x7f176406edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1778006fd0 tx=0x7f1778008040 comp rx=0 tx=0).stop 2024-04-28T12:43:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.805+0000 7f178f800640 1 -- 172.21.15.131:0/1787174271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17880d1250 msgr2=0x7f178815fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.805+0000 7f178f800640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17880d1250 0x7f178815fe40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f177000c400 tx=0x7f177000c8c0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.805+0000 7f178f800640 1 -- 172.21.15.131:0/1787174271 shutdown_connections 2024-04-28T12:43:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.805+0000 7f178f800640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f176406c8f0 0x7f176406edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.805+0000 7f178f800640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f178815cab0 0x7f1788155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.805+0000 7f178f800640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17880d1250 0x7f178815fe40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.805+0000 7f178f800640 1 --2- 172.21.15.131:0/1787174271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17880b0020 0x7f178814d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.805+0000 7f178f800640 1 -- 172.21.15.131:0/1787174271 >> 172.21.15.131:0/1787174271 conn(0x7f17880b2900 msgr2=0x7f178815d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.805+0000 7f178f800640 1 -- 172.21.15.131:0/1787174271 shutdown_connections 2024-04-28T12:43:23.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:23.805+0000 7f178f800640 1 -- 172.21.15.131:0/1787174271 wait complete. 2024-04-28T12:43:23.820 INFO:tasks.ceph:Scrubbing osd.6 2024-04-28T12:43:23.820 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-28T12:43:24.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.022+0000 7fbddb800640 1 Processor -- start 2024-04-28T12:43:24.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.022+0000 7fbddb800640 1 -- start start 2024-04-28T12:43:24.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.024+0000 7fbddb800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbdd40d2ce0 0x7fbdd40d3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.024+0000 7fbddb800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdd40d39b0 0x7fbdd40b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.024+0000 7fbddb800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdd40d0de0 con 0x7fbdd415f1a0 2024-04-28T12:43:24.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.024+0000 7fbddb800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fbdd40b8740 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.024+0000 7fbddb800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd40b7e10 con 0x7fbdd40d39b0 2024-04-28T12:43:24.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.024+0000 7fbdd3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbdd40d2ce0 0x7fbdd40d3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.024+0000 7fbdd3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbdd40d2ce0 0x7fbdd40d3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48758/0 (socket says 172.21.15.131:48758) 2024-04-28T12:43:24.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.024+0000 7fbdd3e00640 1 -- 172.21.15.131:0/2304809599 learned_addr learned my addr 172.21.15.131:0/2304809599 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:24.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.025+0000 7fbdd3e00640 1 -- 172.21.15.131:0/2304809599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdd40d39b0 msgr2=0x7fbdd40b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:24.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.025+0000 7fbdd9800640 1 --2- 172.21.15.131:0/2304809599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdd40d39b0 0x7fbdd40b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.025+0000 7fbdd3e00640 1 --2- 172.21.15.131:0/2304809599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdd40d39b0 0x7fbdd40b6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.025+0000 7fbdd3e00640 1 -- 172.21.15.131:0/2304809599 >> v1:172.21.15.131:6789/0 conn(0x7fbdd415f1a0 legacy=0x7fbdd40d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.025+0000 7fbdd3e00640 1 -- 172.21.15.131:0/2304809599 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd40b80f0 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.025+0000 7fbdd9800640 1 --2- 172.21.15.131:0/2304809599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdd40d39b0 0x7fbdd40b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:24.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.026+0000 7fbdd3e00640 1 --2- 172.21.15.131:0/2304809599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbdd40d2ce0 0x7fbdd40d3470 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009870 tx=0x7fbdc802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b190ca18eb47f98e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.026+0000 7fbdd3400640 1 -- 172.21.15.131:0/2304809599 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8037070 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.026+0000 7fbdd3400640 1 -- 172.21.15.131:0/2304809599 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc8032440 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.026+0000 7fbdd3400640 1 -- 172.21.15.131:0/2304809599 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8032810 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.027+0000 7fbddb800640 1 -- 172.21.15.131:0/2304809599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbdd40d2ce0 msgr2=0x7fbdd40d3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.027+0000 7fbddb800640 1 --2- 172.21.15.131:0/2304809599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbdd40d2ce0 0x7fbdd40d3470 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009870 tx=0x7fbdc802a250 comp rx=0 tx=0).stop 2024-04-28T12:43:24.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.027+0000 7fbddb800640 1 -- 172.21.15.131:0/2304809599 shutdown_connections 2024-04-28T12:43:24.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.027+0000 7fbddb800640 1 --2- 172.21.15.131:0/2304809599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdd40d39b0 0x7fbdd40b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.027+0000 7fbddb800640 1 --2- 172.21.15.131:0/2304809599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbdd40d2ce0 0x7fbdd40d3470 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.027+0000 7fbddb800640 1 -- 172.21.15.131:0/2304809599 >> 172.21.15.131:0/2304809599 conn(0x7fbdd40d4f20 msgr2=0x7fbdd40d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:24.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.027+0000 7fbddb800640 1 -- 172.21.15.131:0/2304809599 shutdown_connections 2024-04-28T12:43:24.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.027+0000 7fbddb800640 1 -- 172.21.15.131:0/2304809599 wait complete. 2024-04-28T12:43:24.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.029+0000 7fbddb800640 1 Processor -- start 2024-04-28T12:43:24.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.029+0000 7fbddb800640 1 -- start start 2024-04-28T12:43:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.030+0000 7fbddb800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbdd40d2ce0 0x7fbdd414e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.030+0000 7fbddb800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdd40d39b0 0x7fbdd40c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.030+0000 7fbddb800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdd415f1a0 0x7fbdd41ead40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.030+0000 7fbddb800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd415f700 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.030+0000 7fbddb800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd415f580 con 0x7fbdd415f1a0 2024-04-28T12:43:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.030+0000 7fbddb800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd40b71a0 con 0x7fbdd40d39b0 2024-04-28T12:43:24.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.030+0000 7fbdd8e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbdd40d2ce0 0x7fbdd414e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.030+0000 7fbdd8e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbdd40d2ce0 0x7fbdd414e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48762/0 (socket says 172.21.15.131:48762) 2024-04-28T12:43:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.030+0000 7fbdd8e00640 1 -- 172.21.15.131:0/2971654304 learned_addr learned my addr 172.21.15.131:0/2971654304 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.030+0000 7fbdd3e00640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdd40d39b0 0x7fbdd40c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.031+0000 7fbdd8e00640 1 -- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdd40d39b0 msgr2=0x7fbdd40c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.031+0000 7fbdd8e00640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdd40d39b0 0x7fbdd40c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.031+0000 7fbdd8e00640 1 -- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdd415f1a0 msgr2=0x7fbdd41ead40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.031+0000 7fbdd8e00640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdd415f1a0 0x7fbdd41ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.031+0000 7fbdd8e00640 1 -- 172.21.15.131:0/2971654304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd4160530 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.031+0000 7fbdd3e00640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdd40d39b0 0x7fbdd40c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:24.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.031+0000 7fbdd8e00640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbdd40d2ce0 0x7fbdd414e3c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c3f0 tx=0x7fbdc000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.031+0000 7fbdd0c00640 1 -- 172.21.15.131:0/2971654304 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00052d0 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.031+0000 7fbdd0c00640 1 -- 172.21.15.131:0/2971654304 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc0005490 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.031+0000 7fbdd0c00640 1 -- 172.21.15.131:0/2971654304 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0007260 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.031+0000 7fbddb800640 1 -- 172.21.15.131:0/2971654304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd4160810 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.031+0000 7fbddb800640 1 -- 172.21.15.131:0/2971654304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd4160dd0 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.032+0000 7fbddb800640 1 -- 172.21.15.131:0/2971654304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbdd4069350 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.033+0000 7fbdd0c00640 1 -- 172.21.15.131:0/2971654304 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbdc0007400 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.034+0000 7fbdd0c00640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbdb406c920 0x7fbdb406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.035+0000 7fbdd0c00640 1 -- 172.21.15.131:0/2971654304 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fbdc008afd0 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.035+0000 7fbdd0c00640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] conn(0x7fbdb4072dc0 0x7fbdb4075200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.035+0000 7fbdd0c00640 1 -- 172.21.15.131:0/2971654304 --> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbdc0009e50 con 0x7fbdb4072dc0 2024-04-28T12:43:24.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.035+0000 7fbdd0c00640 1 -- 172.21.15.131:0/2971654304 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbdc008d5f0 con 0x7fbdd40d2ce0 2024-04-28T12:43:24.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.035+0000 7fbdd3e00640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbdb406c920 0x7fbdb406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.036+0000 7fbdd9800640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] conn(0x7fbdb4072dc0 0x7fbdb4075200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.036+0000 7fbdd3e00640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbdb406c920 0x7fbdb406ede0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8034000 tx=0x7fbdc802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.037+0000 7fbdd9800640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] conn(0x7fbdb4072dc0 0x7fbdb4075200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.039+0000 7fbdd0c00640 1 -- 172.21.15.131:0/2971654304 <== osd.6 v2:172.21.15.157:6810/2594435113 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbdc0009e50 con 0x7fbdb4072dc0 2024-04-28T12:43:24.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.053+0000 7fbddb800640 1 -- 172.21.15.131:0/2971654304 --> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbdd408d090 con 0x7fbdb4072dc0 2024-04-28T12:43:24.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.054+0000 7fbdd0c00640 1 -- 172.21.15.131:0/2971654304 <== osd.6 v2:172.21.15.157:6810/2594435113 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbdd408d090 con 0x7fbdb4072dc0 2024-04-28T12:43:24.051 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:43:24.051 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T12:43:24.051 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:43:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.055+0000 7fbddb800640 1 -- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] conn(0x7fbdb4072dc0 msgr2=0x7fbdb4075200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.055+0000 7fbddb800640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:6810/2594435113,v1:172.21.15.157:6811/2594435113] conn(0x7fbdb4072dc0 0x7fbdb4075200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.055+0000 7fbddb800640 1 -- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbdb406c920 msgr2=0x7fbdb406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.055+0000 7fbddb800640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fbdb406c920 0x7fbdb406ede0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8034000 tx=0x7fbdc802e000 comp rx=0 tx=0).stop 2024-04-28T12:43:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.055+0000 7fbddb800640 1 -- 172.21.15.131:0/2971654304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbdd40d2ce0 msgr2=0x7fbdd414e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.055+0000 7fbddb800640 1 --2- 172.21.15.131:0/2971654304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbdd40d2ce0 0x7fbdd414e3c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c3f0 tx=0x7fbdc000c8b0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.055+0000 7fbdd9800640 1 -- 172.21.15.131:0/2971654304 reap_dead start 2024-04-28T12:43:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.055+0000 7fbddb800640 1 -- 172.21.15.131:0/2971654304 shutdown_connections 2024-04-28T12:43:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.055+0000 7fbddb800640 1 -- 172.21.15.131:0/2971654304 >> 172.21.15.131:0/2971654304 conn(0x7fbdd40d4f20 msgr2=0x7fbdd406d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.055+0000 7fbddb800640 1 -- 172.21.15.131:0/2971654304 shutdown_connections 2024-04-28T12:43:24.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.056+0000 7fbddb800640 1 -- 172.21.15.131:0/2971654304 wait complete. 2024-04-28T12:43:24.069 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-28T12:43:24.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.270+0000 7f6052a00640 1 Processor -- start 2024-04-28T12:43:24.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.270+0000 7f6052a00640 1 -- start start 2024-04-28T12:43:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.272+0000 7f6052a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f604c0b5b80 0x7f604c0d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.272+0000 7f6052a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f604c0d0480 con 0x7f604c0d1620 2024-04-28T12:43:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.272+0000 7f6052a00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f604c0d0720 con 0x7f604c0d2800 2024-04-28T12:43:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.272+0000 7f6052a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f604c0d0ce0 con 0x7f604c0b5b80 2024-04-28T12:43:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.272+0000 7f6051600640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f604c0d1620 0x7f604c0d1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39644/0 (socket says 172.21.15.131:39644) 2024-04-28T12:43:24.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.272+0000 7f6051600640 1 -- 172.21.15.131:0/1579206095 learned_addr learned my addr 172.21.15.131:0/1579206095 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:24.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.273+0000 7f6050c00640 1 --2- 172.21.15.131:0/1579206095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f604c0b5b80 0x7f604c0d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.273+0000 7f604be00640 1 -- 172.21.15.131:0/1579206095 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1537872836 0 0) 0x7f604c0d0480 con 0x7f604c0d1620 2024-04-28T12:43:24.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.273+0000 7f604be00640 1 -- 172.21.15.131:0/1579206095 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6034003520 con 0x7f604c0d1620 2024-04-28T12:43:24.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.273+0000 7f604be00640 1 -- 172.21.15.131:0/1579206095 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2635494037 0 0) 0x7f6034003520 con 0x7f604c0d1620 2024-04-28T12:43:24.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.273+0000 7f6050c00640 1 -- 172.21.15.131:0/1579206095 >> v1:172.21.15.157:6789/0 conn(0x7f604c0d2800 legacy=0x7f604c0d2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.273+0000 7f6050c00640 1 -- 172.21.15.131:0/1579206095 >> v1:172.21.15.131:6789/0 conn(0x7f604c0d1620 legacy=0x7f604c0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.273+0000 7f6050c00640 1 -- 172.21.15.131:0/1579206095 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c0d0a00 con 0x7f604c0b5b80 2024-04-28T12:43:24.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.275+0000 7f6050c00640 1 --2- 172.21.15.131:0/1579206095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f604c0b5b80 0x7f604c0d5750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f603c009dc0 tx=0x7f603c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c48e8cedc379447 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.275+0000 7f604be00640 1 -- 172.21.15.131:0/1579206095 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040001670 con 0x7f604c0b5b80 2024-04-28T12:43:24.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.275+0000 7f604be00640 1 -- 172.21.15.131:0/1579206095 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603c007540 con 0x7f604c0b5b80 2024-04-28T12:43:24.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.275+0000 7f604be00640 1 -- 172.21.15.131:0/1579206095 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c0078f0 con 0x7f604c0b5b80 2024-04-28T12:43:24.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.276+0000 7f6052a00640 1 -- 172.21.15.131:0/1579206095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f604c0b5b80 msgr2=0x7f604c0d5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.276+0000 7f6052a00640 1 --2- 172.21.15.131:0/1579206095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f604c0b5b80 0x7f604c0d5750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f603c009dc0 tx=0x7f603c02a5d0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.276+0000 7f6052a00640 1 -- 172.21.15.131:0/1579206095 shutdown_connections 2024-04-28T12:43:24.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.276+0000 7f6052a00640 1 --2- 172.21.15.131:0/1579206095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f604c0b5b80 0x7f604c0d5750 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.276+0000 7f6052a00640 1 -- 172.21.15.131:0/1579206095 >> 172.21.15.131:0/1579206095 conn(0x7f604c0d4c30 msgr2=0x7f604c0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:24.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.276+0000 7f6052a00640 1 -- 172.21.15.131:0/1579206095 shutdown_connections 2024-04-28T12:43:24.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.276+0000 7f6052a00640 1 -- 172.21.15.131:0/1579206095 wait complete. 2024-04-28T12:43:24.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.278+0000 7f6052a00640 1 Processor -- start 2024-04-28T12:43:24.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.278+0000 7f6052a00640 1 -- start start 2024-04-28T12:43:24.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.279+0000 7f6052a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f604c0b5b80 0x7f604c0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.279+0000 7f6052a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f604c0d1620 0x7f604c14cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.279+0000 7f6052a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f604c0d2800 0x7f604c0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.279+0000 7f6052a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f604c0d3520 con 0x7f604c0d2800 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.279+0000 7f6052a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f604c0d33a0 con 0x7f604c0b5b80 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.279+0000 7f6052a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f604c0d36a0 con 0x7f604c0d1620 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.279+0000 7f6052000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f604c0d2800 0x7f604c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.279+0000 7f6052000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f604c0d2800 0x7f604c0c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48774/0 (socket says 172.21.15.131:48774) 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.279+0000 7f6052000640 1 -- 172.21.15.131:0/1484347833 learned_addr learned my addr 172.21.15.131:0/1484347833 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.280+0000 7f6051600640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f604c0b5b80 0x7f604c0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.280+0000 7f6050c00640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f604c0d1620 0x7f604c14cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.280+0000 7f6052000640 1 -- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f604c0d1620 msgr2=0x7f604c14cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.280+0000 7f6052000640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f604c0d1620 0x7f604c14cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.280+0000 7f6052000640 1 -- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f604c0b5b80 msgr2=0x7f604c0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.280+0000 7f6052000640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f604c0b5b80 0x7f604c0cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.280+0000 7f6052000640 1 -- 172.21.15.131:0/1484347833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c1f8ec0 con 0x7f604c0d2800 2024-04-28T12:43:24.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.280+0000 7f6052000640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f604c0d2800 0x7f604c0c8590 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f604400c390 tx=0x7f604400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.280+0000 7f6049600640 1 -- 172.21.15.131:0/1484347833 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044017070 con 0x7f604c0d2800 2024-04-28T12:43:24.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.280+0000 7f6049600640 1 -- 172.21.15.131:0/1484347833 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60440052d0 con 0x7f604c0d2800 2024-04-28T12:43:24.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.280+0000 7f6049600640 1 -- 172.21.15.131:0/1484347833 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044003810 con 0x7f604c0d2800 2024-04-28T12:43:24.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.280+0000 7f6052a00640 1 -- 172.21.15.131:0/1484347833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604c1f8620 con 0x7f604c0d2800 2024-04-28T12:43:24.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.281+0000 7f6052a00640 1 -- 172.21.15.131:0/1484347833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604c1f8be0 con 0x7f604c0d2800 2024-04-28T12:43:24.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.282+0000 7f6052a00640 1 -- 172.21.15.131:0/1484347833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f604c069350 con 0x7f604c0d2800 2024-04-28T12:43:24.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.283+0000 7f6049600640 1 -- 172.21.15.131:0/1484347833 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f604401a070 con 0x7f604c0d2800 2024-04-28T12:43:24.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.284+0000 7f6049600640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f603006c8f0 0x7f603006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.284+0000 7f6049600640 1 -- 172.21.15.131:0/1484347833 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f604408acf0 con 0x7f604c0d2800 2024-04-28T12:43:24.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.284+0000 7f6051600640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f603006c8f0 0x7f603006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.285+0000 7f6051600640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f603006c8f0 0x7f603006edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6040005880 tx=0x7f6040009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.288+0000 7f6049600640 1 -- 172.21.15.131:0/1484347833 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6044059280 con 0x7f604c0d2800 2024-04-28T12:43:24.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.491+0000 7f6052a00640 1 -- 172.21.15.131:0/1484347833 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f604c0ca4e0 con 0x7f603006c8f0 2024-04-28T12:43:24.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.492+0000 7f6049600640 1 -- 172.21.15.131:0/1484347833 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f604c0ca4e0 con 0x7f603006c8f0 2024-04-28T12:43:24.488 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 6 to deep-scrub 2024-04-28T12:43:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.495+0000 7f6052a00640 1 -- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f603006c8f0 msgr2=0x7f603006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.495+0000 7f6052a00640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f603006c8f0 0x7f603006edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6040005880 tx=0x7f6040009040 comp rx=0 tx=0).stop 2024-04-28T12:43:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.495+0000 7f6052a00640 1 -- 172.21.15.131:0/1484347833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f604c0d2800 msgr2=0x7f604c0c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.495+0000 7f6052a00640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f604c0d2800 0x7f604c0c8590 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f604400c390 tx=0x7f604400c850 comp rx=0 tx=0).stop 2024-04-28T12:43:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.495+0000 7f6052a00640 1 -- 172.21.15.131:0/1484347833 shutdown_connections 2024-04-28T12:43:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.495+0000 7f6052a00640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f603006c8f0 0x7f603006edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.495+0000 7f6052a00640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f604c0d2800 0x7f604c0c8590 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.495+0000 7f6052a00640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f604c0d1620 0x7f604c14cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.495+0000 7f6052a00640 1 --2- 172.21.15.131:0/1484347833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f604c0b5b80 0x7f604c0cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.495+0000 7f6052a00640 1 -- 172.21.15.131:0/1484347833 >> 172.21.15.131:0/1484347833 conn(0x7f604c0d4c30 msgr2=0x7f604c06dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.495+0000 7f6052a00640 1 -- 172.21.15.131:0/1484347833 shutdown_connections 2024-04-28T12:43:24.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.496+0000 7f6052a00640 1 -- 172.21.15.131:0/1484347833 wait complete. 2024-04-28T12:43:24.511 INFO:tasks.ceph:Scrubbing osd.7 2024-04-28T12:43:24.512 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-28T12:43:24.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.721+0000 7fdee7c00640 1 Processor -- start 2024-04-28T12:43:24.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.721+0000 7fdee7c00640 1 -- start start 2024-04-28T12:43:24.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.723+0000 7fdee7c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00d30e0 0x7fdee00b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.723+0000 7fdee7c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee00d0ff0 con 0x7fdee015f410 2024-04-28T12:43:24.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.723+0000 7fdee7c00640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee00d12d0 con 0x7fdee00d2c80 2024-04-28T12:43:24.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.723+0000 7fdee7c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdee00d0dc0 con 0x7fdee00d30e0 2024-04-28T12:43:24.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.723+0000 7fdee5200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdee015f410 0x7fdee006de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39658/0 (socket says 172.21.15.131:39658) 2024-04-28T12:43:24.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.723+0000 7fdee5200640 1 -- 172.21.15.131:0/3042172323 learned_addr learned my addr 172.21.15.131:0/3042172323 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:24.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 410762931 0 0) 0x7fdee00d0ff0 con 0x7fdee015f410 2024-04-28T12:43:24.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdec4003520 con 0x7fdee015f410 2024-04-28T12:43:24.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdee5c00640 1 --2- 172.21.15.131:0/3042172323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00d30e0 0x7fdee00b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3202934368 0 0) 0x7fdec4003520 con 0x7fdee015f410 2024-04-28T12:43:24.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00d30e0 msgr2=0x7fdee00b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:24.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 --2- 172.21.15.131:0/3042172323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00d30e0 0x7fdee00b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 >> v1:172.21.15.157:6789/0 conn(0x7fdee00d2c80 legacy=0x7fdee00b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee00d15b0 con 0x7fdee015f410 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7fded0001670 con 0x7fdee015f410 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 >> v1:172.21.15.131:6789/0 conn(0x7fdee015f410 legacy=0x7fdee006de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 --2- 172.21.15.131:0/3042172323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec40039e0 0x7fdec4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 --2- 172.21.15.131:0/3042172323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00d30e0 0x7fdec4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 --2- 172.21.15.131:0/3042172323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdec4004d40 0x7fdec40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdec400bb50 con 0x7fdec40039e0 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdec400bcd0 con 0x7fdec4004d40 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.724+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdec400bea0 con 0x7fdee00d30e0 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.725+0000 7fdee5200640 1 --2- 172.21.15.131:0/3042172323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec40039e0 0x7fdec4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.725+0000 7fded7e00640 1 --2- 172.21.15.131:0/3042172323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00d30e0 0x7fdec4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.725+0000 7fdee5200640 1 -- 172.21.15.131:0/3042172323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00d30e0 msgr2=0x7fdec4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.725+0000 7fdee5200640 1 --2- 172.21.15.131:0/3042172323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00d30e0 0x7fdec4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.725+0000 7fdee5200640 1 -- 172.21.15.131:0/3042172323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdec4004d40 msgr2=0x7fdec40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:24.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.725+0000 7fdee5200640 1 --2- 172.21.15.131:0/3042172323 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdec4004d40 0x7fdec40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.725+0000 7fdee5200640 1 -- 172.21.15.131:0/3042172323 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec4003520 con 0x7fdec40039e0 2024-04-28T12:43:24.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.725+0000 7fded7e00640 1 --2- 172.21.15.131:0/3042172323 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee00d30e0 0x7fdec4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:24.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.726+0000 7fdee5200640 1 --2- 172.21.15.131:0/3042172323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec40039e0 0x7fdec4003e00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fded000c240 tx=0x7fdec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6d13fd92c69783 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.726+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0007d30 con 0x7fdec40039e0 2024-04-28T12:43:24.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.726+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded0005af0 con 0x7fdec40039e0 2024-04-28T12:43:24.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.726+0000 7fdedfe00640 1 -- 172.21.15.131:0/3042172323 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0007070 con 0x7fdec40039e0 2024-04-28T12:43:24.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.727+0000 7fdee7c00640 1 -- 172.21.15.131:0/3042172323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec40039e0 msgr2=0x7fdec4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.727+0000 7fdee7c00640 1 --2- 172.21.15.131:0/3042172323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec40039e0 0x7fdec4003e00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fded000c240 tx=0x7fdec4005830 comp rx=0 tx=0).stop 2024-04-28T12:43:24.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.727+0000 7fdee5c00640 1 -- 172.21.15.131:0/3042172323 reap_dead start 2024-04-28T12:43:24.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.727+0000 7fdee7c00640 1 -- 172.21.15.131:0/3042172323 shutdown_connections 2024-04-28T12:43:24.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.727+0000 7fdee7c00640 1 -- 172.21.15.131:0/3042172323 >> 172.21.15.131:0/3042172323 conn(0x7fdee009ba50 msgr2=0x7fdee009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:24.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.727+0000 7fdee7c00640 1 -- 172.21.15.131:0/3042172323 shutdown_connections 2024-04-28T12:43:24.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.727+0000 7fdee7c00640 1 -- 172.21.15.131:0/3042172323 wait complete. 2024-04-28T12:43:24.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.729+0000 7fdee7c00640 1 Processor -- start 2024-04-28T12:43:24.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.729+0000 7fdee7c00640 1 -- start start 2024-04-28T12:43:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.730+0000 7fdee7c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdec40039e0 0x7fdee00c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.730+0000 7fdee7c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee01f8ac0 0x7fdee01f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.730+0000 7fdee7c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee00c7b90 0x7fdee015cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.730+0000 7fdee7c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdee00d3550 con 0x7fdee00c7b90 2024-04-28T12:43:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.730+0000 7fdee7c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdee00d1b70 con 0x7fdec40039e0 2024-04-28T12:43:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.730+0000 7fdee7c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdee00b2350 con 0x7fdee01f8ac0 2024-04-28T12:43:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.730+0000 7fdee5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee00c7b90 0x7fdee015cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.730+0000 7fdee5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee00c7b90 0x7fdee015cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48798/0 (socket says 172.21.15.131:48798) 2024-04-28T12:43:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.730+0000 7fdee5c00640 1 -- 172.21.15.131:0/2204202410 learned_addr learned my addr 172.21.15.131:0/2204202410 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fdee5200640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdec40039e0 0x7fdee00c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fded7e00640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee01f8ac0 0x7fdee01f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fdee5c00640 1 -- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee01f8ac0 msgr2=0x7fdee01f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fdee5c00640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdee01f8ac0 0x7fdee01f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fdee5c00640 1 -- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdec40039e0 msgr2=0x7fdee00c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fdee5c00640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdec40039e0 0x7fdee00c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fdee5c00640 1 -- 172.21.15.131:0/2204202410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee01f8fd0 con 0x7fdee00c7b90 2024-04-28T12:43:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fdee5200640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdec40039e0 0x7fdee00c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fdee5c00640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee00c7b90 0x7fdee015cdd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdee0076210 tx=0x7fded800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fdedd600640 1 -- 172.21.15.131:0/2204202410 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded800ebc0 con 0x7fdee00c7b90 2024-04-28T12:43:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fdedd600640 1 -- 172.21.15.131:0/2204202410 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded800ed80 con 0x7fdee00c7b90 2024-04-28T12:43:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fdedd600640 1 -- 172.21.15.131:0/2204202410 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8003bd0 con 0x7fdee00c7b90 2024-04-28T12:43:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.731+0000 7fdee7c00640 1 -- 172.21.15.131:0/2204202410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee01f92b0 con 0x7fdee00c7b90 2024-04-28T12:43:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.732+0000 7fdee7c00640 1 -- 172.21.15.131:0/2204202410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee01f9870 con 0x7fdee00c7b90 2024-04-28T12:43:24.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.733+0000 7fdee7c00640 1 -- 172.21.15.131:0/2204202410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdee0162540 con 0x7fdee00c7b90 2024-04-28T12:43:24.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.734+0000 7fdedd600640 1 -- 172.21.15.131:0/2204202410 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fded801b020 con 0x7fdee00c7b90 2024-04-28T12:43:24.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.735+0000 7fdedd600640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdebc06c920 0x7fdebc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.735+0000 7fdedd600640 1 -- 172.21.15.131:0/2204202410 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fded808b830 con 0x7fdee00c7b90 2024-04-28T12:43:24.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.735+0000 7fdedd600640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] conn(0x7fdebc072dc0 0x7fdebc075200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.735+0000 7fdedd600640 1 -- 172.21.15.131:0/2204202410 --> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fded8020e80 con 0x7fdebc072dc0 2024-04-28T12:43:24.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.735+0000 7fdedd600640 1 -- 172.21.15.131:0/2204202410 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fded8092050 con 0x7fdee00c7b90 2024-04-28T12:43:24.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.735+0000 7fdee5200640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdebc06c920 0x7fdebc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.736+0000 7fded7e00640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] conn(0x7fdebc072dc0 0x7fdebc075200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.737+0000 7fded7e00640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] conn(0x7fdebc072dc0 0x7fdebc075200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.737+0000 7fdee5200640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdebc06c920 0x7fdebc06ede0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fded0033000 tx=0x7fded0009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.739+0000 7fdedd600640 1 -- 172.21.15.131:0/2204202410 <== osd.7 v2:172.21.15.157:6821/2246043568 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fded8020e80 con 0x7fdebc072dc0 2024-04-28T12:43:24.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.753+0000 7fdee7c00640 1 -- 172.21.15.131:0/2204202410 --> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdee0069350 con 0x7fdebc072dc0 2024-04-28T12:43:24.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.754+0000 7fdedd600640 1 -- 172.21.15.131:0/2204202410 <== osd.7 v2:172.21.15.157:6821/2246043568 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdee0069350 con 0x7fdebc072dc0 2024-04-28T12:43:24.751 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-04-28T12:43:24.751 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-28T12:43:24.751 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-04-28T12:43:24.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.755+0000 7fdee7c00640 1 -- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] conn(0x7fdebc072dc0 msgr2=0x7fdebc075200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.755+0000 7fdee7c00640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:6821/2246043568,v1:172.21.15.157:6823/2246043568] conn(0x7fdebc072dc0 0x7fdebc075200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.755+0000 7fdee7c00640 1 -- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdebc06c920 msgr2=0x7fdebc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.755+0000 7fdee7c00640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fdebc06c920 0x7fdebc06ede0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fded0033000 tx=0x7fded0009030 comp rx=0 tx=0).stop 2024-04-28T12:43:24.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.755+0000 7fdee7c00640 1 -- 172.21.15.131:0/2204202410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee00c7b90 msgr2=0x7fdee015cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.755+0000 7fdee7c00640 1 --2- 172.21.15.131:0/2204202410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdee00c7b90 0x7fdee015cdd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdee0076210 tx=0x7fded800e4b0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.755+0000 7fdee5c00640 1 -- 172.21.15.131:0/2204202410 reap_dead start 2024-04-28T12:43:24.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.755+0000 7fdee7c00640 1 -- 172.21.15.131:0/2204202410 shutdown_connections 2024-04-28T12:43:24.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.755+0000 7fdee7c00640 1 -- 172.21.15.131:0/2204202410 >> 172.21.15.131:0/2204202410 conn(0x7fdee009ba50 msgr2=0x7fdee0070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:24.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.756+0000 7fdee7c00640 1 -- 172.21.15.131:0/2204202410 shutdown_connections 2024-04-28T12:43:24.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.756+0000 7fdee7c00640 1 -- 172.21.15.131:0/2204202410 wait complete. 2024-04-28T12:43:24.770 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-28T12:43:24.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.977+0000 7f9ab5600640 1 Processor -- start 2024-04-28T12:43:24.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.978+0000 7f9ab5600640 1 -- start start 2024-04-28T12:43:24.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.979+0000 7f9ab5600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9ab00b71a0 0x7f9ab00d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.979+0000 7f9ab5600640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f9ab00b7580 0x7f9ab015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.979+0000 7f9ab5600640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab00d3a10 con 0x7f9ab015ef00 2024-04-28T12:43:24.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.979+0000 7f9ab5600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab00c8830 con 0x7f9ab00b71a0 2024-04-28T12:43:24.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.979+0000 7f9ab5600640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab00d3cb0 con 0x7f9ab00b7580 2024-04-28T12:43:24.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.979+0000 7f9aaea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9ab00b71a0 0x7f9ab00d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.979+0000 7f9aaea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9ab00b71a0 0x7f9ab00d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48814/0 (socket says 172.21.15.131:48814) 2024-04-28T12:43:24.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.979+0000 7f9aaea00640 1 -- 172.21.15.131:0/2966659966 learned_addr learned my addr 172.21.15.131:0/2966659966 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:24.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.980+0000 7f9aaea00640 1 -- 172.21.15.131:0/2966659966 >> v2:172.21.15.157:3300/0 conn(0x7f9ab00b7580 msgr2=0x7f9ab015e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:24.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.980+0000 7f9aaea00640 1 --2- 172.21.15.131:0/2966659966 >> v2:172.21.15.157:3300/0 conn(0x7f9ab00b7580 0x7f9ab015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.980+0000 7f9aaea00640 1 -- 172.21.15.131:0/2966659966 >> v1:172.21.15.157:6789/0 conn(0x7f9ab015ef00 legacy=0x7f9ab0162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:24.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.980+0000 7f9aaea00640 1 -- 172.21.15.131:0/2966659966 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab00c8af0 con 0x7f9ab00b71a0 2024-04-28T12:43:24.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.981+0000 7f9aaea00640 1 --2- 172.21.15.131:0/2966659966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab00b71a0 0x7f9ab00d34d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9ab008d0b0 tx=0x7f9aa002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bff1636201bb438 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.981+0000 7f9aad600640 1 -- 172.21.15.131:0/2966659966 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0037070 con 0x7f9ab00b71a0 2024-04-28T12:43:24.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.981+0000 7f9aad600640 1 -- 172.21.15.131:0/2966659966 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa0032440 con 0x7f9ab00b71a0 2024-04-28T12:43:24.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.981+0000 7f9aad600640 1 -- 172.21.15.131:0/2966659966 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0032810 con 0x7f9ab00b71a0 2024-04-28T12:43:24.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.982+0000 7f9ab5600640 1 -- 172.21.15.131:0/2966659966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab00b71a0 msgr2=0x7f9ab00d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.982+0000 7f9ab5600640 1 --2- 172.21.15.131:0/2966659966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab00b71a0 0x7f9ab00d34d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9ab008d0b0 tx=0x7f9aa002a1c0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.982+0000 7f9ab5600640 1 -- 172.21.15.131:0/2966659966 shutdown_connections 2024-04-28T12:43:24.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.982+0000 7f9ab5600640 1 --2- 172.21.15.131:0/2966659966 >> v2:172.21.15.157:3300/0 conn(0x7f9ab00b7580 0x7f9ab015e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.982+0000 7f9ab5600640 1 --2- 172.21.15.131:0/2966659966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab00b71a0 0x7f9ab00d34d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.982+0000 7f9ab5600640 1 -- 172.21.15.131:0/2966659966 >> 172.21.15.131:0/2966659966 conn(0x7f9ab00d4d70 msgr2=0x7f9ab00d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:24.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.982+0000 7f9ab5600640 1 -- 172.21.15.131:0/2966659966 shutdown_connections 2024-04-28T12:43:24.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.982+0000 7f9ab5600640 1 -- 172.21.15.131:0/2966659966 wait complete. 2024-04-28T12:43:24.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.984+0000 7f9ab5600640 1 Processor -- start 2024-04-28T12:43:24.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.984+0000 7f9ab5600640 1 -- start start 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.985+0000 7f9ab5600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ab00b71a0 0x7f9ab01f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.985+0000 7f9ab5600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ab00b7580 0x7f9ab0156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.985+0000 7f9ab5600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015ef00 0x7f9ab01574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.985+0000 7f9ab5600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab015f330 con 0x7f9ab015ef00 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.985+0000 7f9ab5600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab00c90b0 con 0x7f9ab00b7580 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.985+0000 7f9ab5600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab015f4b0 con 0x7f9ab00b71a0 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.985+0000 7f9aaf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015ef00 0x7f9ab01574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.985+0000 7f9aaf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015ef00 0x7f9ab01574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48826/0 (socket says 172.21.15.131:48826) 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.985+0000 7f9aaf400640 1 -- 172.21.15.131:0/2974849950 learned_addr learned my addr 172.21.15.131:0/2974849950 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aaea00640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ab00b71a0 0x7f9ab01f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aae000640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ab00b7580 0x7f9ab0156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aaf400640 1 -- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ab00b71a0 msgr2=0x7f9ab01f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aaf400640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ab00b71a0 0x7f9ab01f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aaf400640 1 -- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ab00b7580 msgr2=0x7f9ab0156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:24.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aaf400640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ab00b7580 0x7f9ab0156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:24.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aaf400640 1 -- 172.21.15.131:0/2974849950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab00cc970 con 0x7f9ab015ef00 2024-04-28T12:43:24.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aaea00640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ab00b71a0 0x7f9ab01f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:24.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aae000640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ab00b7580 0x7f9ab0156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:24.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aaf400640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015ef00 0x7f9ab01574f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800b490 tx=0x7f9aa800b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aa6a00640 1 -- 172.21.15.131:0/2974849950 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8015020 con 0x7f9ab015ef00 2024-04-28T12:43:24.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aa6a00640 1 -- 172.21.15.131:0/2974849950 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa8011850 con 0x7f9ab015ef00 2024-04-28T12:43:24.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9aa6a00640 1 -- 172.21.15.131:0/2974849950 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8011c40 con 0x7f9ab015ef00 2024-04-28T12:43:24.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.986+0000 7f9ab5600640 1 -- 172.21.15.131:0/2974849950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab00ccc50 con 0x7f9ab015ef00 2024-04-28T12:43:24.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.987+0000 7f9ab5600640 1 -- 172.21.15.131:0/2974849950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab00b9950 con 0x7f9ab015ef00 2024-04-28T12:43:24.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.988+0000 7f9ab5600640 1 -- 172.21.15.131:0/2974849950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab0069350 con 0x7f9ab015ef00 2024-04-28T12:43:24.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.989+0000 7f9aa6a00640 1 -- 172.21.15.131:0/2974849950 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9aa8018020 con 0x7f9ab015ef00 2024-04-28T12:43:24.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.990+0000 7f9aa6a00640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:24.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.990+0000 7f9aa6a00640 1 -- 172.21.15.131:0/2974849950 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f9aa808b8b0 con 0x7f9ab015ef00 2024-04-28T12:43:24.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.991+0000 7f9aaea00640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:24.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.992+0000 7f9aaea00640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0034000 tx=0x7f9aa002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:24.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:24.995+0000 7f9aa6a00640 1 -- 172.21.15.131:0/2974849950 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9aa801f020 con 0x7f9ab015ef00 2024-04-28T12:43:25.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.200+0000 7f9ab5600640 1 -- 172.21.15.131:0/2974849950 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f9ab008ee00 con 0x7f9a8c06c8f0 2024-04-28T12:43:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.201+0000 7f9aa6a00640 1 -- 172.21.15.131:0/2974849950 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9ab008ee00 con 0x7f9a8c06c8f0 2024-04-28T12:43:25.197 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 7 to deep-scrub 2024-04-28T12:43:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.203+0000 7f9ab5600640 1 -- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9a8c06c8f0 msgr2=0x7f9a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.203+0000 7f9ab5600640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0034000 tx=0x7f9aa002e000 comp rx=0 tx=0).stop 2024-04-28T12:43:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.203+0000 7f9ab5600640 1 -- 172.21.15.131:0/2974849950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015ef00 msgr2=0x7f9ab01574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.203+0000 7f9ab5600640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015ef00 0x7f9ab01574f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800b490 tx=0x7f9aa800b950 comp rx=0 tx=0).stop 2024-04-28T12:43:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.203+0000 7f9ab5600640 1 -- 172.21.15.131:0/2974849950 shutdown_connections 2024-04-28T12:43:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.203+0000 7f9ab5600640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.203+0000 7f9ab5600640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015ef00 0x7f9ab01574f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.203+0000 7f9ab5600640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ab00b7580 0x7f9ab0156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.203+0000 7f9ab5600640 1 --2- 172.21.15.131:0/2974849950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ab00b71a0 0x7f9ab01f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.203+0000 7f9ab5600640 1 -- 172.21.15.131:0/2974849950 >> 172.21.15.131:0/2974849950 conn(0x7f9ab00d4d70 msgr2=0x7f9ab00c95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:25.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.203+0000 7f9ab5600640 1 -- 172.21.15.131:0/2974849950 shutdown_connections 2024-04-28T12:43:25.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.204+0000 7f9ab5600640 1 -- 172.21.15.131:0/2974849950 wait complete. 2024-04-28T12:43:25.218 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:43:25.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.405+0000 7f54c7a00640 1 Processor -- start 2024-04-28T12:43:25.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.405+0000 7f54c7a00640 1 -- start start 2024-04-28T12:43:25.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.407+0000 7f54c7a00640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f54c00d0b40 0x7f54c00d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:25.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.407+0000 7f54c7a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015f5c0 0x7f54c015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:25.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.407+0000 7f54c7a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54c00b0650 con 0x7f54c015cab0 2024-04-28T12:43:25.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.407+0000 7f54c7a00640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f54c00d0900 con 0x7f54c00d0b40 2024-04-28T12:43:25.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.407+0000 7f54c7a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f54c01635a0 con 0x7f54c015f5c0 2024-04-28T12:43:25.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.407+0000 7f54c5000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f54c015cab0 0x7f54c00afae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39674/0 (socket says 172.21.15.131:39674) 2024-04-28T12:43:25.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.407+0000 7f54c5000640 1 -- 172.21.15.131:0/3035928294 learned_addr learned my addr 172.21.15.131:0/3035928294 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:25.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.407+0000 7f54b7e00640 1 --2- 172.21.15.131:0/3035928294 >> v2:172.21.15.157:3300/0 conn(0x7f54c00d0b40 0x7f54c00d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.407+0000 7f54c5a00640 1 --2- 172.21.15.131:0/3035928294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015f5c0 0x7f54c015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2683174644 0 0) 0x7f54c00b0650 con 0x7f54c015cab0 2024-04-28T12:43:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54a4003520 con 0x7f54c015cab0 2024-04-28T12:43:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2829904336 0 0) 0x7f54a4003520 con 0x7f54c015cab0 2024-04-28T12:43:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015f5c0 msgr2=0x7f54c015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 --2- 172.21.15.131:0/3035928294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015f5c0 0x7f54c015fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 >> v2:172.21.15.157:3300/0 conn(0x7f54c00d0b40 msgr2=0x7f54c00d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 --2- 172.21.15.131:0/3035928294 >> v2:172.21.15.157:3300/0 conn(0x7f54c00d0b40 0x7f54c00d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c0163a30 con 0x7f54c015cab0 2024-04-28T12:43:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1163010115 0 0) 0x7f54b0001670 con 0x7f54c015cab0 2024-04-28T12:43:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54b7e00640 1 --2- 172.21.15.131:0/3035928294 >> v2:172.21.15.157:3300/0 conn(0x7f54c00d0b40 0x7f54c00d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:43:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 >> v1:172.21.15.131:6789/0 conn(0x7f54c015cab0 legacy=0x7f54c00afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 --2- 172.21.15.131:0/3035928294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015f5c0 0x7f54a40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 --2- 172.21.15.131:0/3035928294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54a4003f30 0x7f54a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 --2- 172.21.15.131:0/3035928294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54a4004d40 0x7f54a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01635a0 con 0x7f54a4003f30 2024-04-28T12:43:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54a400b9c0 con 0x7f54a4004d40 2024-04-28T12:43:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f54a400bb40 con 0x7f54c015f5c0 2024-04-28T12:43:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.408+0000 7f54b7e00640 1 --2- 172.21.15.131:0/3035928294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54a4003f30 0x7f54a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.409+0000 7f54c5000640 1 --2- 172.21.15.131:0/3035928294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015f5c0 0x7f54a40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.409+0000 7f54c5a00640 1 --2- 172.21.15.131:0/3035928294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54a4004d40 0x7f54a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.409+0000 7f54b7e00640 1 -- 172.21.15.131:0/3035928294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015f5c0 msgr2=0x7f54a40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.409+0000 7f54b7e00640 1 --2- 172.21.15.131:0/3035928294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c015f5c0 0x7f54a40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.409+0000 7f54b7e00640 1 -- 172.21.15.131:0/3035928294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54a4004d40 msgr2=0x7f54a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:25.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.409+0000 7f54b7e00640 1 --2- 172.21.15.131:0/3035928294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54a4004d40 0x7f54a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.409+0000 7f54b7e00640 1 -- 172.21.15.131:0/3035928294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a4003520 con 0x7f54a4003f30 2024-04-28T12:43:25.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.410+0000 7f54b7e00640 1 --2- 172.21.15.131:0/3035928294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54a4003f30 0x7f54a4004800 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009c90 tx=0x7f54a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c1a13a877aed13 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:25.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.410+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8036070 con 0x7f54a4003f30 2024-04-28T12:43:25.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.410+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a8031400 con 0x7f54a4003f30 2024-04-28T12:43:25.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.410+0000 7f54bfe00640 1 -- 172.21.15.131:0/3035928294 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8009860 con 0x7f54a4003f30 2024-04-28T12:43:25.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.411+0000 7f54c7a00640 1 -- 172.21.15.131:0/3035928294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54a4003f30 msgr2=0x7f54a4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:25.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.411+0000 7f54c7a00640 1 --2- 172.21.15.131:0/3035928294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54a4003f30 0x7f54a4004800 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009c90 tx=0x7f54a4005830 comp rx=0 tx=0).stop 2024-04-28T12:43:25.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.411+0000 7f54c5a00640 1 -- 172.21.15.131:0/3035928294 reap_dead start 2024-04-28T12:43:25.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.411+0000 7f54c7a00640 1 -- 172.21.15.131:0/3035928294 shutdown_connections 2024-04-28T12:43:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.411+0000 7f54c7a00640 1 -- 172.21.15.131:0/3035928294 >> 172.21.15.131:0/3035928294 conn(0x7f54c00b2900 msgr2=0x7f54c00b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.411+0000 7f54c7a00640 1 -- 172.21.15.131:0/3035928294 shutdown_connections 2024-04-28T12:43:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.411+0000 7f54c7a00640 1 -- 172.21.15.131:0/3035928294 wait complete. 2024-04-28T12:43:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.413+0000 7f54c7a00640 1 Processor -- start 2024-04-28T12:43:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.413+0000 7f54c7a00640 1 -- start start 2024-04-28T12:43:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54c7a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54a4003f30 0x7f54c015b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54c7a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54c014f970 0x7f54c01f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54c7a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c014fdf0 0x7f54c006ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54c7a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54c0162f50 con 0x7f54c014f970 2024-04-28T12:43:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54c7a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54c00d14c0 con 0x7f54a4003f30 2024-04-28T12:43:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54c7a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f54c0163230 con 0x7f54c014fdf0 2024-04-28T12:43:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54c014f970 0x7f54c01f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54c014f970 0x7f54c01f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48854/0 (socket says 172.21.15.131:48854) 2024-04-28T12:43:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54b7e00640 1 -- 172.21.15.131:0/2513442892 learned_addr learned my addr 172.21.15.131:0/2513442892 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54c5000640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54a4003f30 0x7f54c015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:25.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54b7e00640 1 -- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c014fdf0 msgr2=0x7f54c006ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:25.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54b7e00640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c014fdf0 0x7f54c006ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54b7e00640 1 -- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54a4003f30 msgr2=0x7f54c015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:25.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54b7e00640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54a4003f30 0x7f54c015b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54b7e00640 1 -- 172.21.15.131:0/2513442892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c01f8c80 con 0x7f54c014f970 2024-04-28T12:43:25.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.414+0000 7f54c5000640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54a4003f30 0x7f54c015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:25.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.415+0000 7f54b7e00640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54c014f970 0x7f54c01f81c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009c90 tx=0x7f54a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:25.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.415+0000 7f54bd600640 1 -- 172.21.15.131:0/2513442892 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8040020 con 0x7f54c014f970 2024-04-28T12:43:25.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.415+0000 7f54bd600640 1 -- 172.21.15.131:0/2513442892 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a803b680 con 0x7f54c014f970 2024-04-28T12:43:25.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.415+0000 7f54bd600640 1 -- 172.21.15.131:0/2513442892 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8036040 con 0x7f54c014f970 2024-04-28T12:43:25.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.415+0000 7f54c7a00640 1 -- 172.21.15.131:0/2513442892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c01f8f60 con 0x7f54c014f970 2024-04-28T12:43:25.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.415+0000 7f54c7a00640 1 -- 172.21.15.131:0/2513442892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c015a630 con 0x7f54c014f970 2024-04-28T12:43:25.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.416+0000 7f54c7a00640 1 -- 172.21.15.131:0/2513442892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c0069350 con 0x7f54c014f970 2024-04-28T12:43:25.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.417+0000 7f54bd600640 1 -- 172.21.15.131:0/2513442892 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54a8043020 con 0x7f54c014f970 2024-04-28T12:43:25.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.418+0000 7f54bd600640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f549c06c8f0 0x7f549c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:25.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.418+0000 7f54bd600640 1 -- 172.21.15.131:0/2513442892 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f54a80ac110 con 0x7f54c014f970 2024-04-28T12:43:25.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.419+0000 7f54c5000640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f549c06c8f0 0x7f549c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:25.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.420+0000 7f54c5000640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f549c06c8f0 0x7f549c06edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f54b0001050 tx=0x7f54b0000fe0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:25.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.423+0000 7f54bd600640 1 -- 172.21.15.131:0/2513442892 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f54a804a030 con 0x7f54c014f970 2024-04-28T12:43:25.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.617+0000 7f54c7a00640 1 -- 172.21.15.131:0/2513442892 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54c015cf30 con 0x7f549c06c8f0 2024-04-28T12:43:25.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.619+0000 7f54bd600640 1 -- 172.21.15.131:0/2513442892 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13353 (secure 0 0 0) 0x7f54c015cf30 con 0x7f549c06c8f0 2024-04-28T12:43:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:25.616 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:43:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.622+0000 7f54c7a00640 1 -- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f549c06c8f0 msgr2=0x7f549c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.622+0000 7f54c7a00640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f549c06c8f0 0x7f549c06edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f54b0001050 tx=0x7f54b0000fe0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.622+0000 7f54c7a00640 1 -- 172.21.15.131:0/2513442892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54c014f970 msgr2=0x7f54c01f81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.622+0000 7f54c7a00640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54c014f970 0x7f54c01f81c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009c90 tx=0x7f54a8004730 comp rx=0 tx=0).stop 2024-04-28T12:43:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.622+0000 7f54c7a00640 1 -- 172.21.15.131:0/2513442892 shutdown_connections 2024-04-28T12:43:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.622+0000 7f54c7a00640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f549c06c8f0 0x7f549c06edb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.622+0000 7f54c7a00640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54c014fdf0 0x7f54c006ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.622+0000 7f54c7a00640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54c014f970 0x7f54c01f81c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.622+0000 7f54c7a00640 1 --2- 172.21.15.131:0/2513442892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54a4003f30 0x7f54c015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.622+0000 7f54c7a00640 1 -- 172.21.15.131:0/2513442892 >> 172.21.15.131:0/2513442892 conn(0x7f54c00b2900 msgr2=0x7f54c00b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.622+0000 7f54c7a00640 1 -- 172.21.15.131:0/2513442892 shutdown_connections 2024-04-28T12:43:25.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:25.623+0000 7f54c7a00640 1 -- 172.21.15.131:0/2513442892 wait complete. 2024-04-28T12:43:25.638 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-28T12:43:25.060927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237948,"kb_used_data":5128,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736772,"statfs":{"total":773094113280,"available":772850454528,"internally_reserved":0,"allocated":5251072,"data_stored":2953800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.099860"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-28T12:43:15.025839+0000","last_change":"2024-04-28T12:37:16.983253+0000","last_active":"2024-04-28T12:43:15.025839+0000","last_peered":"2024-04-28T12:43:15.025839+0000","last_clean":"2024-04-28T12:43:15.025839+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:43:15.025839+0000","last_undegraded":"2024-04-28T12:43:15.025839+0000","last_fullsized":"2024-04-28T12:43:15.025839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:37:15.664620+0000","last_clean_scrub_stamp":"2024-04-28T12:37:15.664620+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:45:23.325066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":812001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":221633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":221633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":221633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27520,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":507904,"data_stored":221633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27520,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":507904,"data_stored":221633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":221633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1101824,"data_stored":812001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:43:25.641 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-28T12:37:15.664620+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=12, tm_min=37, tm_sec=15, tm_wday=6, tm_yday=119, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=28, tm_hour=12, tm_min=43, tm_sec=18, tm_wday=6, tm_yday=119, tm_isdst=0) 2024-04-28T12:43:25.641 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-28T12:43:45.642 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:43:45.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.859+0000 7f7897000640 1 Processor -- start 2024-04-28T12:43:45.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.859+0000 7f7897000640 1 -- start start 2024-04-28T12:43:45.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.860+0000 7f7897000640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7f7890072b70 0x7f78900d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:45.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.860+0000 7f7897000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f78900d1060 0x7f789006dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:45.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.861+0000 7f7897000640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7890146ba0 con 0x7f78901467c0 2024-04-28T12:43:45.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.861+0000 7f7897000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7890073dd0 con 0x7f78900d1060 2024-04-28T12:43:45.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.861+0000 7f7897000640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7f78900731c0 con 0x7f7890072b70 2024-04-28T12:43:45.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.861+0000 7f7895000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f78900d1060 0x7f789006dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:45.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.861+0000 7f7895000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f78900d1060 0x7f789006dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57644/0 (socket says 172.21.15.131:57644) 2024-04-28T12:43:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.861+0000 7f7895000640 1 -- 172.21.15.131:0/4080539407 learned_addr learned my addr 172.21.15.131:0/4080539407 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.861+0000 7f7887400640 1 --2- 172.21.15.131:0/4080539407 >> v2:172.21.15.157:3300/0 conn(0x7f7890072b70 0x7f78900d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.861+0000 7f7895000640 1 -- 172.21.15.131:0/4080539407 >> v2:172.21.15.157:3300/0 conn(0x7f7890072b70 msgr2=0x7f78900d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.861+0000 7f7895000640 1 --2- 172.21.15.131:0/4080539407 >> v2:172.21.15.157:3300/0 conn(0x7f7890072b70 0x7f78900d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.861+0000 7f7895000640 1 -- 172.21.15.131:0/4080539407 >> v1:172.21.15.157:6789/0 conn(0x7f78901467c0 legacy=0x7f78900b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.861+0000 7f7895000640 1 -- 172.21.15.131:0/4080539407 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78900734a0 con 0x7f78900d1060 2024-04-28T12:43:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.862+0000 7f7895000640 1 --2- 172.21.15.131:0/4080539407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78900d1060 0x7f789006dff0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7890072320 tx=0x7f788802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b2eafb35fa6a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.863+0000 7f788f400640 1 -- 172.21.15.131:0/4080539407 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7888036070 con 0x7f78900d1060 2024-04-28T12:43:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.863+0000 7f788f400640 1 -- 172.21.15.131:0/4080539407 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7888031600 con 0x7f78900d1060 2024-04-28T12:43:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.863+0000 7f788f400640 1 -- 172.21.15.131:0/4080539407 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78880319d0 con 0x7f78900d1060 2024-04-28T12:43:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.863+0000 7f7897000640 1 -- 172.21.15.131:0/4080539407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78900d1060 msgr2=0x7f789006dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.863+0000 7f7897000640 1 --2- 172.21.15.131:0/4080539407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78900d1060 0x7f789006dff0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7890072320 tx=0x7f788802a6f0 comp rx=0 tx=0).stop 2024-04-28T12:43:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.863+0000 7f7897000640 1 -- 172.21.15.131:0/4080539407 shutdown_connections 2024-04-28T12:43:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.863+0000 7f7897000640 1 --2- 172.21.15.131:0/4080539407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78900d1060 0x7f789006dff0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.863+0000 7f7897000640 1 --2- 172.21.15.131:0/4080539407 >> v2:172.21.15.157:3300/0 conn(0x7f7890072b70 0x7f78900d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.863+0000 7f7897000640 1 -- 172.21.15.131:0/4080539407 >> 172.21.15.131:0/4080539407 conn(0x7f78900b2900 msgr2=0x7f78900b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.864+0000 7f7897000640 1 -- 172.21.15.131:0/4080539407 shutdown_connections 2024-04-28T12:43:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.864+0000 7f7897000640 1 -- 172.21.15.131:0/4080539407 wait complete. 2024-04-28T12:43:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.865+0000 7f7897000640 1 Processor -- start 2024-04-28T12:43:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.865+0000 7f7897000640 1 -- start start 2024-04-28T12:43:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.866+0000 7f7897000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7890072b70 0x7f789014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.866+0000 7f7897000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78900d1060 0x7f789013bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:45.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.866+0000 7f7897000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78901467c0 0x7f789013c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.866+0000 7f7897000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f789006e530 con 0x7f7890072b70 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.866+0000 7f7897000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7890073a60 con 0x7f78900d1060 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.866+0000 7f7897000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f789006e6b0 con 0x7f78901467c0 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.867+0000 7f788fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7890072b70 0x7f789014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.867+0000 7f788fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7890072b70 0x7f789014b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57656/0 (socket says 172.21.15.131:57656) 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.867+0000 7f788fe00640 1 -- 172.21.15.131:0/3772577538 learned_addr learned my addr 172.21.15.131:0/3772577538 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.867+0000 7f7887400640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78900d1060 0x7f789013bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.867+0000 7f7895000640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78901467c0 0x7f789013c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.867+0000 7f788fe00640 1 -- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78901467c0 msgr2=0x7f789013c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.867+0000 7f788fe00640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78901467c0 0x7f789013c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.867+0000 7f788fe00640 1 -- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78900d1060 msgr2=0x7f789013bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.867+0000 7f788fe00640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78900d1060 0x7f789013bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.867+0000 7f788fe00640 1 -- 172.21.15.131:0/3772577538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78901401d0 con 0x7f7890072b70 2024-04-28T12:43:45.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.867+0000 7f788fe00640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7890072b70 0x7f789014b2c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f788000b420 tx=0x7f788000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:45.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.868+0000 7f788cc00640 1 -- 172.21.15.131:0/3772577538 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880011830 con 0x7f7890072b70 2024-04-28T12:43:45.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.868+0000 7f788cc00640 1 -- 172.21.15.131:0/3772577538 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7880016070 con 0x7f7890072b70 2024-04-28T12:43:45.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.868+0000 7f788cc00640 1 -- 172.21.15.131:0/3772577538 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880011c10 con 0x7f7890072b70 2024-04-28T12:43:45.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.868+0000 7f7897000640 1 -- 172.21.15.131:0/3772577538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78901404b0 con 0x7f7890072b70 2024-04-28T12:43:45.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.868+0000 7f7897000640 1 -- 172.21.15.131:0/3772577538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7890140a70 con 0x7f7890072b70 2024-04-28T12:43:45.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.869+0000 7f7897000640 1 -- 172.21.15.131:0/3772577538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7890069350 con 0x7f7890072b70 2024-04-28T12:43:45.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.870+0000 7f788cc00640 1 -- 172.21.15.131:0/3772577538 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f788001b460 con 0x7f7890072b70 2024-04-28T12:43:45.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.871+0000 7f788cc00640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f786c06c8f0 0x7f786c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:45.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.872+0000 7f788cc00640 1 -- 172.21.15.131:0/3772577538 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f788008b570 con 0x7f7890072b70 2024-04-28T12:43:45.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.872+0000 7f7887400640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f786c06c8f0 0x7f786c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:45.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.873+0000 7f7887400640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f786c06c8f0 0x7f786c06edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f787800a970 tx=0x7f7878008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:45.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:45.876+0000 7f788cc00640 1 -- 172.21.15.131:0/3772577538 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7880020020 con 0x7f7890072b70 2024-04-28T12:43:46.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.072+0000 7f7897000640 1 -- 172.21.15.131:0/3772577538 --> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78900b1b40 con 0x7f786c06c8f0 2024-04-28T12:43:46.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.075+0000 7f788cc00640 1 -- 172.21.15.131:0/3772577538 <== mgr.4102 v2:172.21.15.157:6800/2340731527 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13360 (secure 0 0 0) 0x7f78900b1b40 con 0x7f786c06c8f0 2024-04-28T12:43:46.071 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:46.071 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-04-28T12:43:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.077+0000 7f7897000640 1 -- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f786c06c8f0 msgr2=0x7f786c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.077+0000 7f7897000640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f786c06c8f0 0x7f786c06edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f787800a970 tx=0x7f7878008040 comp rx=0 tx=0).stop 2024-04-28T12:43:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.077+0000 7f7897000640 1 -- 172.21.15.131:0/3772577538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7890072b70 msgr2=0x7f789014b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.077+0000 7f7897000640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7890072b70 0x7f789014b2c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f788000b420 tx=0x7f788000b8e0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.077+0000 7f7897000640 1 -- 172.21.15.131:0/3772577538 shutdown_connections 2024-04-28T12:43:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.077+0000 7f7897000640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f786c06c8f0 0x7f786c06edb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.077+0000 7f7897000640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78901467c0 0x7f789013c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.077+0000 7f7897000640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78900d1060 0x7f789013bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.077+0000 7f7897000640 1 --2- 172.21.15.131:0/3772577538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7890072b70 0x7f789014b2c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.077+0000 7f7897000640 1 -- 172.21.15.131:0/3772577538 >> 172.21.15.131:0/3772577538 conn(0x7f78900b2900 msgr2=0x7f78900b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:46.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.077+0000 7f7897000640 1 -- 172.21.15.131:0/3772577538 shutdown_connections 2024-04-28T12:43:46.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.078+0000 7f7897000640 1 -- 172.21.15.131:0/3772577538 wait complete. 2024-04-28T12:43:46.093 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-04-28T12:43:45.066644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237940,"kb_used_data":5128,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736780,"statfs":{"total":773094113280,"available":772850462720,"internally_reserved":0,"allocated":5251072,"data_stored":2953800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003354"},"pg_stats":[{"pgid":"1.0","version":"43'119","reported_seq":227,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-28T12:43:25.880175+0000","last_change":"2024-04-28T12:43:25.880277+0000","last_active":"2024-04-28T12:43:25.880175+0000","last_peered":"2024-04-28T12:43:25.880175+0000","last_clean":"2024-04-28T12:43:25.880175+0000","last_became_active":"2024-04-28T12:37:16.980983+0000","last_became_peered":"2024-04-28T12:37:16.980983+0000","last_unstale":"2024-04-28T12:43:25.880175+0000","last_undegraded":"2024-04-28T12:43:25.880175+0000","last_fullsized":"2024-04-28T12:43:25.880175+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'119","last_scrub_stamp":"2024-04-28T12:43:25.880009+0000","last_deep_scrub":"43'119","last_deep_scrub_stamp":"2024-04-28T12:43:25.880009+0000","last_clean_scrub_stamp":"2024-04-28T12:43:25.880009+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:34:38.957715+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1101824,"data_stored":812001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":221633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":221633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":221633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27520,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":507904,"data_stored":221633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27520,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":507904,"data_stored":221633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":221633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1101824,"data_stored":812001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:43:46.093 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-28T12:43:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.269+0000 7fc822200640 1 Processor -- start 2024-04-28T12:43:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.269+0000 7fc822200640 1 -- start start 2024-04-28T12:43:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.271+0000 7fc822200640 1 --2- >> v2:172.21.15.157:3300/0 conn(0x7fc81c15f180 0x7fc81c0d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.271+0000 7fc822200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81c0b3cb0 0x7fc81c0d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.271+0000 7fc822200640 1 -- --> v1:172.21.15.157:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc81c0d2d70 con 0x7fc81c0d3660 2024-04-28T12:43:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.271+0000 7fc822200640 1 -- --> v2:172.21.15.157:3300/0 -- mon_getmap magic: 0 -- 0x7fc81c15f560 con 0x7fc81c15f180 2024-04-28T12:43:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.271+0000 7fc822200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc81c0b2750 con 0x7fc81c0b3cb0 2024-04-28T12:43:46.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.272+0000 7fc81be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81c0b3cb0 0x7fc81c0d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:46.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.272+0000 7fc81be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81c0b3cb0 0x7fc81c0d3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.131:43342/0 (socket says 172.21.15.131:43342) 2024-04-28T12:43:46.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.272+0000 7fc81be00640 1 -- 172.21.15.131:0/2410323242 learned_addr learned my addr 172.21.15.131:0/2410323242 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.272+0000 7fc820e00640 1 --2- 172.21.15.131:0/2410323242 >> v2:172.21.15.157:3300/0 conn(0x7fc81c15f180 0x7fc81c0d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.273+0000 7fc81be00640 1 -- 172.21.15.131:0/2410323242 >> v2:172.21.15.157:3300/0 conn(0x7fc81c15f180 msgr2=0x7fc81c0d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.273+0000 7fc81be00640 1 --2- 172.21.15.131:0/2410323242 >> v2:172.21.15.157:3300/0 conn(0x7fc81c15f180 0x7fc81c0d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.273+0000 7fc81be00640 1 -- 172.21.15.131:0/2410323242 >> v1:172.21.15.157:6789/0 conn(0x7fc81c0d3660 legacy=0x7fc81c0d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.273+0000 7fc81be00640 1 -- 172.21.15.131:0/2410323242 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81c0d06e0 con 0x7fc81c0b3cb0 2024-04-28T12:43:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.274+0000 7fc81be00640 1 --2- 172.21.15.131:0/2410323242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81c0b3cb0 0x7fc81c0d3120 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc804009870 tx=0x7fc80402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=557a756c5f7dd524 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:46.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.274+0000 7fc81b400640 1 -- 172.21.15.131:0/2410323242 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804037070 con 0x7fc81c0b3cb0 2024-04-28T12:43:46.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.275+0000 7fc81b400640 1 -- 172.21.15.131:0/2410323242 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc804032440 con 0x7fc81c0b3cb0 2024-04-28T12:43:46.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.275+0000 7fc81b400640 1 -- 172.21.15.131:0/2410323242 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804032810 con 0x7fc81c0b3cb0 2024-04-28T12:43:46.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.275+0000 7fc822200640 1 -- 172.21.15.131:0/2410323242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81c0b3cb0 msgr2=0x7fc81c0d3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.275+0000 7fc822200640 1 --2- 172.21.15.131:0/2410323242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81c0b3cb0 0x7fc81c0d3120 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc804009870 tx=0x7fc80402a250 comp rx=0 tx=0).stop 2024-04-28T12:43:46.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.275+0000 7fc822200640 1 -- 172.21.15.131:0/2410323242 shutdown_connections 2024-04-28T12:43:46.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.275+0000 7fc822200640 1 --2- 172.21.15.131:0/2410323242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81c0b3cb0 0x7fc81c0d3120 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.275+0000 7fc822200640 1 --2- 172.21.15.131:0/2410323242 >> v2:172.21.15.157:3300/0 conn(0x7fc81c15f180 0x7fc81c0d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.275+0000 7fc822200640 1 -- 172.21.15.131:0/2410323242 >> 172.21.15.131:0/2410323242 conn(0x7fc81c0d4bd0 msgr2=0x7fc81c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:46.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.275+0000 7fc822200640 1 -- 172.21.15.131:0/2410323242 shutdown_connections 2024-04-28T12:43:46.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.275+0000 7fc822200640 1 -- 172.21.15.131:0/2410323242 wait complete. 2024-04-28T12:43:46.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.277+0000 7fc822200640 1 Processor -- start 2024-04-28T12:43:46.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.277+0000 7fc822200640 1 -- start start 2024-04-28T12:43:46.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.278+0000 7fc822200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81c0b3cb0 0x7fc81c0b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:46.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.278+0000 7fc822200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0d3660 0x7fc81c1eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.278+0000 7fc822200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc81c15f180 0x7fc81c1f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.278+0000 7fc822200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c0d2a40 con 0x7fc81c15f180 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.278+0000 7fc822200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c0d0ca0 con 0x7fc81c0d3660 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.278+0000 7fc822200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc81c0d11c0 con 0x7fc81c0b3cb0 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.278+0000 7fc821800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc81c15f180 0x7fc81c1f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.278+0000 7fc821800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc81c15f180 0x7fc81c1f4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57666/0 (socket says 172.21.15.131:57666) 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.278+0000 7fc821800640 1 -- 172.21.15.131:0/970538236 learned_addr learned my addr 172.21.15.131:0/970538236 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.279+0000 7fc81be00640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0d3660 0x7fc81c1eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.279+0000 7fc821800640 1 -- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81c0b3cb0 msgr2=0x7fc81c0b64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.279+0000 7fc821800640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81c0b3cb0 0x7fc81c0b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.279+0000 7fc821800640 1 -- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0d3660 msgr2=0x7fc81c1eeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.279+0000 7fc821800640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0d3660 0x7fc81c1eeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.279+0000 7fc821800640 1 -- 172.21.15.131:0/970538236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81c0ca020 con 0x7fc81c15f180 2024-04-28T12:43:46.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.279+0000 7fc821800640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc81c15f180 0x7fc81c1f4c50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc814001eb0 tx=0x7fc81400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.279+0000 7fc818c00640 1 -- 172.21.15.131:0/970538236 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814017070 con 0x7fc81c15f180 2024-04-28T12:43:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.279+0000 7fc818c00640 1 -- 172.21.15.131:0/970538236 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8140052d0 con 0x7fc81c15f180 2024-04-28T12:43:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.279+0000 7fc822200640 1 -- 172.21.15.131:0/970538236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81c0ca300 con 0x7fc81c15f180 2024-04-28T12:43:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.279+0000 7fc818c00640 1 -- 172.21.15.131:0/970538236 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814003810 con 0x7fc81c15f180 2024-04-28T12:43:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.279+0000 7fc822200640 1 -- 172.21.15.131:0/970538236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc81c0ca8c0 con 0x7fc81c15f180 2024-04-28T12:43:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.280+0000 7fc822200640 1 -- 172.21.15.131:0/970538236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc81c0506c0 con 0x7fc81c15f180 2024-04-28T12:43:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.281+0000 7fc818c00640 1 -- 172.21.15.131:0/970538236 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc81401a070 con 0x7fc81c15f180 2024-04-28T12:43:46.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.282+0000 7fc818c00640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7f806c8f0 0x7fc7f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:46.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.282+0000 7fc818c00640 1 -- 172.21.15.131:0/970538236 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7fc81408ade0 con 0x7fc81c15f180 2024-04-28T12:43:46.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.283+0000 7fc820e00640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7f806c8f0 0x7fc7f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:46.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.284+0000 7fc820e00640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7f806c8f0 0x7fc7f806edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0070b0 tx=0x7fc80c007040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:46.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.287+0000 7fc818c00640 1 -- 172.21.15.131:0/970538236 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc8140592e0 con 0x7fc81c15f180 2024-04-28T12:43:46.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.471+0000 7fc822200640 1 -- 172.21.15.131:0/970538236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fc81c06dbe0 con 0x7fc81c15f180 2024-04-28T12:43:46.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.479+0000 7fc818c00640 1 -- 172.21.15.131:0/970538236 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7fc814058be0 con 0x7fc81c15f180 2024-04-28T12:43:46.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.481+0000 7fc818c00640 1 -- 172.21.15.131:0/970538236 <== mon.0 v2:172.21.15.131:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc81408e030 con 0x7fc81c15f180 2024-04-28T12:43:46.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.483+0000 7fc822200640 1 -- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7f806c8f0 msgr2=0x7fc7f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.483+0000 7fc822200640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7f806c8f0 0x7fc7f806edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0070b0 tx=0x7fc80c007040 comp rx=0 tx=0).stop 2024-04-28T12:43:46.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.483+0000 7fc822200640 1 -- 172.21.15.131:0/970538236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc81c15f180 msgr2=0x7fc81c1f4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.483+0000 7fc822200640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc81c15f180 0x7fc81c1f4c50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc814001eb0 tx=0x7fc81400c890 comp rx=0 tx=0).stop 2024-04-28T12:43:46.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.483+0000 7fc822200640 1 -- 172.21.15.131:0/970538236 shutdown_connections 2024-04-28T12:43:46.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.483+0000 7fc822200640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7fc7f806c8f0 0x7fc7f806edb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.483+0000 7fc822200640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc81c15f180 0x7fc81c1f4c50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.483+0000 7fc822200640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc81c0d3660 0x7fc81c1eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.483+0000 7fc822200640 1 --2- 172.21.15.131:0/970538236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc81c0b3cb0 0x7fc81c0b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.483+0000 7fc822200640 1 -- 172.21.15.131:0/970538236 >> 172.21.15.131:0/970538236 conn(0x7fc81c0d4bd0 msgr2=0x7fc81c0cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:46.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.484+0000 7fc822200640 1 -- 172.21.15.131:0/970538236 shutdown_connections 2024-04-28T12:43:46.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.484+0000 7fc822200640 1 -- 172.21.15.131:0/970538236 wait complete. 2024-04-28T12:43:46.497 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-28T12:43:46.497 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:43:46.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.688+0000 7f22e7c00640 1 Processor -- start 2024-04-28T12:43:46.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.688+0000 7f22e7c00640 1 -- start start 2024-04-28T12:43:46.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.690+0000 7f22e7c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22e0148970 0x7f22e015f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.690+0000 7f22e7c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f22e015fb30 0x7f22e00b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.690+0000 7f22e7c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22e015ffd0 con 0x7f22e006c430 2024-04-28T12:43:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.690+0000 7f22e7c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f22e0160590 con 0x7f22e015fb30 2024-04-28T12:43:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.690+0000 7f22e7c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f22e006f3b0 con 0x7f22e0148970 2024-04-28T12:43:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.690+0000 7f22e5c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f22e006c430 0x7f22e0072eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:53706/0 (socket says 172.21.15.131:53706) 2024-04-28T12:43:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.690+0000 7f22e5c00640 1 -- 172.21.15.131:0/4290459795 learned_addr learned my addr 172.21.15.131:0/4290459795 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-04-28T12:43:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.690+0000 7f22d7e00640 1 --2- 172.21.15.131:0/4290459795 >> v2:172.21.15.131:3300/0 conn(0x7f22e015fb30 0x7f22e00b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:46.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.691+0000 7f22d7e00640 1 -- 172.21.15.131:0/4290459795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22e0148970 msgr2=0x7f22e015f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:43:46.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.691+0000 7f22e5200640 1 --2- 172.21.15.131:0/4290459795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22e0148970 0x7f22e015f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.691+0000 7f22d7e00640 1 --2- 172.21.15.131:0/4290459795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22e0148970 0x7f22e015f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.691+0000 7f22dfe00640 1 -- 172.21.15.131:0/4290459795 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 742574146 0 0) 0x7f22e015ffd0 con 0x7f22e006c430 2024-04-28T12:43:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.691+0000 7f22d7e00640 1 -- 172.21.15.131:0/4290459795 >> v1:172.21.15.131:6789/0 conn(0x7f22e006c430 legacy=0x7f22e0072eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.691+0000 7f22d7e00640 1 -- 172.21.15.131:0/4290459795 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e01602b0 con 0x7f22e015fb30 2024-04-28T12:43:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.691+0000 7f22e5200640 1 --2- 172.21.15.131:0/4290459795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22e0148970 0x7f22e015f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:43:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.692+0000 7f22d7e00640 1 --2- 172.21.15.131:0/4290459795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e015fb30 0x7f22e00b4c10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f22c80097a0 tx=0x7f22c802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1ec8bc57a0c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.692+0000 7f22dfe00640 1 -- 172.21.15.131:0/4290459795 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8037070 con 0x7f22e015fb30 2024-04-28T12:43:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.692+0000 7f22dfe00640 1 -- 172.21.15.131:0/4290459795 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f22c802adb0 con 0x7f22e015fb30 2024-04-28T12:43:46.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.692+0000 7f22dfe00640 1 -- 172.21.15.131:0/4290459795 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8032680 con 0x7f22e015fb30 2024-04-28T12:43:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.693+0000 7f22e7c00640 1 -- 172.21.15.131:0/4290459795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e015fb30 msgr2=0x7f22e00b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.693+0000 7f22e7c00640 1 --2- 172.21.15.131:0/4290459795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e015fb30 0x7f22e00b4c10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f22c80097a0 tx=0x7f22c802a060 comp rx=0 tx=0).stop 2024-04-28T12:43:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.693+0000 7f22e7c00640 1 -- 172.21.15.131:0/4290459795 shutdown_connections 2024-04-28T12:43:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.693+0000 7f22e7c00640 1 --2- 172.21.15.131:0/4290459795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e015fb30 0x7f22e00b4c10 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.693+0000 7f22e7c00640 1 --2- 172.21.15.131:0/4290459795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22e0148970 0x7f22e015f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.693+0000 7f22e7c00640 1 -- 172.21.15.131:0/4290459795 >> 172.21.15.131:0/4290459795 conn(0x7f22e00b3910 msgr2=0x7f22e00b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.693+0000 7f22e7c00640 1 -- 172.21.15.131:0/4290459795 shutdown_connections 2024-04-28T12:43:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.693+0000 7f22e7c00640 1 -- 172.21.15.131:0/4290459795 wait complete. 2024-04-28T12:43:46.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.695+0000 7f22e7c00640 1 Processor -- start 2024-04-28T12:43:46.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.695+0000 7f22e7c00640 1 -- start start 2024-04-28T12:43:46.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.696+0000 7f22e7c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22e006c430 0x7f22e015ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.696+0000 7f22e7c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22e0148970 0x7f22e00cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.696+0000 7f22e7c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e015fb30 0x7f22e0146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.696+0000 7f22e7c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f22e00709d0 con 0x7f22e015fb30 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.696+0000 7f22e7c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f22e0070850 con 0x7f22e0148970 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.696+0000 7f22e7c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f22e0070b50 con 0x7f22e006c430 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.696+0000 7f22e5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e015fb30 0x7f22e0146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.696+0000 7f22e5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e015fb30 0x7f22e0146820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57688/0 (socket says 172.21.15.131:57688) 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.696+0000 7f22e5c00640 1 -- 172.21.15.131:0/3122769078 learned_addr learned my addr 172.21.15.131:0/3122769078 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.696+0000 7f22e5200640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22e006c430 0x7f22e015ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.696+0000 7f22d7e00640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22e0148970 0x7f22e00cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.697+0000 7f22e5c00640 1 -- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22e006c430 msgr2=0x7f22e015ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.697+0000 7f22e5c00640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22e006c430 0x7f22e015ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.697+0000 7f22e5c00640 1 -- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22e0148970 msgr2=0x7f22e00cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.697+0000 7f22e5c00640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22e0148970 0x7f22e00cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.697+0000 7f22e5c00640 1 -- 172.21.15.131:0/3122769078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e015efb0 con 0x7f22e015fb30 2024-04-28T12:43:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.697+0000 7f22e5c00640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e015fb30 0x7f22e0146820 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c730 tx=0x7f22d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.697+0000 7f22dd600640 1 -- 172.21.15.131:0/3122769078 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8018070 con 0x7f22e015fb30 2024-04-28T12:43:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.697+0000 7f22dd600640 1 -- 172.21.15.131:0/3122769078 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f22d800ce40 con 0x7f22e015fb30 2024-04-28T12:43:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.697+0000 7f22e7c00640 1 -- 172.21.15.131:0/3122769078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e00c80b0 con 0x7f22e015fb30 2024-04-28T12:43:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.697+0000 7f22dd600640 1 -- 172.21.15.131:0/3122769078 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d800a880 con 0x7f22e015fb30 2024-04-28T12:43:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.697+0000 7f22e7c00640 1 -- 172.21.15.131:0/3122769078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e00c8630 con 0x7f22e015fb30 2024-04-28T12:43:46.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.698+0000 7f22e7c00640 1 -- 172.21.15.131:0/3122769078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e0069a60 con 0x7f22e015fb30 2024-04-28T12:43:46.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.699+0000 7f22dd600640 1 -- 172.21.15.131:0/3122769078 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22d801b070 con 0x7f22e015fb30 2024-04-28T12:43:46.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.700+0000 7f22dd600640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f22bc06c8f0 0x7f22bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:46.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.700+0000 7f22dd600640 1 -- 172.21.15.131:0/3122769078 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5335+0+0 (secure 0 0 0) 0x7f22d808b0c0 con 0x7f22e015fb30 2024-04-28T12:43:46.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.701+0000 7f22e5200640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f22bc06c8f0 0x7f22bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:46.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.702+0000 7f22e5200640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f22bc06c8f0 0x7f22bc06edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f22d000a990 tx=0x7f22d0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:46.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.705+0000 7f22dd600640 1 -- 172.21.15.131:0/3122769078 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22d8059530 con 0x7f22e015fb30 2024-04-28T12:43:46.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.952+0000 7f22e7c00640 1 -- 172.21.15.131:0/3122769078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22e0073e10 con 0x7f22e015fb30 2024-04-28T12:43:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.953+0000 7f22dd600640 1 -- 172.21.15.131:0/3122769078 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3872 (secure 0 0 0) 0x7f22d8058e30 con 0x7f22e015fb30 2024-04-28T12:43:46.949 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-04-28T12:43:46.949 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/361318591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":361318591},{"type":"v1","addr":"172.21.15.157:6836","nonce":361318591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/102335005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":102335005},{"type":"v1","addr":"172.21.15.131:6835","nonce":102335005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4954,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/451171856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":451171856},{"type":"v1","addr":"172.21.15.157:6837","nonce":451171856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1609059776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1609059776},{"type":"v1","addr":"172.21.15.131:6833","nonce":1609059776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-28T12:43:46.949 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 19 2024-04-28T12:43:46.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.955+0000 7f22e7c00640 1 -- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f22bc06c8f0 msgr2=0x7f22bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.955+0000 7f22e7c00640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f22bc06c8f0 0x7f22bc06edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f22d000a990 tx=0x7f22d0008040 comp rx=0 tx=0).stop 2024-04-28T12:43:46.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.955+0000 7f22e7c00640 1 -- 172.21.15.131:0/3122769078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e015fb30 msgr2=0x7f22e0146820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:46.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.955+0000 7f22e7c00640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e015fb30 0x7f22e0146820 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c730 tx=0x7f22d800cbf0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.956+0000 7f22e7c00640 1 -- 172.21.15.131:0/3122769078 shutdown_connections 2024-04-28T12:43:46.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.956+0000 7f22e7c00640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:6800/2340731527,v1:172.21.15.157:6801/2340731527] conn(0x7f22bc06c8f0 0x7f22bc06edb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.956+0000 7f22e7c00640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22e015fb30 0x7f22e0146820 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.956+0000 7f22e7c00640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22e0148970 0x7f22e00cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.956+0000 7f22e7c00640 1 --2- 172.21.15.131:0/3122769078 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22e006c430 0x7f22e015ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:46.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.956+0000 7f22e7c00640 1 -- 172.21.15.131:0/3122769078 >> 172.21.15.131:0/3122769078 conn(0x7f22e00b3910 msgr2=0x7f22e00d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:46.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.956+0000 7f22e7c00640 1 -- 172.21.15.131:0/3122769078 shutdown_connections 2024-04-28T12:43:46.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-04-28T12:43:46.956+0000 7f22e7c00640 1 -- 172.21.15.131:0/3122769078 wait complete. 2024-04-28T12:43:46.968 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:43:46.973 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:43:46.974 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-28T12:43:46.975 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-28T12:43:46.975 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-28T12:43:46.975 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.033 INFO:tasks.ceph.mds.a:Stopped 2024-04-28T12:43:47.033 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-28T12:43:47.033 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.099 INFO:tasks.ceph.mds.c:Stopped 2024-04-28T12:43:47.099 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-28T12:43:47.099 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.126 INFO:tasks.ceph.mds.b:Stopped 2024-04-28T12:43:47.126 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-28T12:43:47.126 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.194 INFO:tasks.ceph.mds.d:Stopped 2024-04-28T12:43:47.194 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-28T12:43:47.195 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-28T12:43:47.195 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.249 INFO:tasks.ceph.osd.0:Stopped 2024-04-28T12:43:47.249 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-28T12:43:47.249 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.314 INFO:tasks.ceph.osd.1:Stopped 2024-04-28T12:43:47.315 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-28T12:43:47.315 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.371 INFO:tasks.ceph.osd.2:Stopped 2024-04-28T12:43:47.372 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-28T12:43:47.372 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.436 INFO:tasks.ceph.osd.3:Stopped 2024-04-28T12:43:47.436 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-28T12:43:47.436 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.481 INFO:tasks.ceph.osd.4:Stopped 2024-04-28T12:43:47.481 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-28T12:43:47.481 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.552 INFO:tasks.ceph.osd.5:Stopped 2024-04-28T12:43:47.552 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-28T12:43:47.552 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.626 INFO:tasks.ceph.osd.6:Stopped 2024-04-28T12:43:47.626 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-28T12:43:47.626 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.688 INFO:tasks.ceph.osd.7:Stopped 2024-04-28T12:43:47.688 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-28T12:43:47.688 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-28T12:43:47.688 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.774 INFO:tasks.ceph.mgr.y:Stopped 2024-04-28T12:43:47.774 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-28T12:43:47.774 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.858 INFO:tasks.ceph.mgr.x:Stopped 2024-04-28T12:43:47.858 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-28T12:43:47.859 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-28T12:43:47.859 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.910 INFO:tasks.ceph.mon.a:Stopped 2024-04-28T12:43:47.910 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-28T12:43:47.910 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:47.943 INFO:tasks.ceph.mon.b:Stopped 2024-04-28T12:43:47.943 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-28T12:43:47.943 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:43:48.018 INFO:tasks.ceph.mon.c:Stopped 2024-04-28T12:43:48.018 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-28T12:43:48.018 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-04-28T12:43:48.124 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:43:48.124 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-28T12:43:48.449 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:43:48.449 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-28T12:43:48.582 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:43:48.582 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-28T12:43:48.711 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:43:48.711 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-28T12:43:48.872 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:43:48.872 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-28T12:43:49.114 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:43:49.115 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-28T12:43:49.263 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:43:49.263 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-28T12:43:49.393 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:43:49.393 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-28T12:43:49.529 INFO:tasks.ceph:Archiving mon data... 2024-04-28T12:43:49.530 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878/data/mon.a.tgz 2024-04-28T12:43:49.530 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-04-28T12:43:49.586 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.gDuluSOUKl 2024-04-28T12:43:49.586 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.gDuluSOUKl 2024-04-28T12:43:49.806 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.gDuluSOUKl 2024-04-28T12:43:49.938 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.gDuluSOUKl is 350KB 2024-04-28T12:43:50.045 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.gDuluSOUKl 2024-04-28T12:43:50.102 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878/data/mon.b.tgz 2024-04-28T12:43:50.102 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-04-28T12:43:50.160 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.vTPJvtNZ0s 2024-04-28T12:43:50.160 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.vTPJvtNZ0s 2024-04-28T12:43:50.365 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.vTPJvtNZ0s 2024-04-28T12:43:50.503 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.vTPJvtNZ0s is 462KB 2024-04-28T12:43:50.611 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.vTPJvtNZ0s 2024-04-28T12:43:50.668 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878/data/mon.c.tgz 2024-04-28T12:43:50.668 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-04-28T12:43:50.718 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.w454POfkEt 2024-04-28T12:43:50.718 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.w454POfkEt 2024-04-28T12:43:50.902 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.w454POfkEt 2024-04-28T12:43:51.031 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.w454POfkEt is 460KB 2024-04-28T12:43:51.142 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.w454POfkEt 2024-04-28T12:43:51.198 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-28T12:43:51.198 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-28T12:43:51.206 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-28T12:43:51.344 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-28T12:43:51.400 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-28T12:43:51.400 INFO:tasks.ceph:Archiving crash dumps... 2024-04-28T12:43:51.402 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878/remote/smithi131/crash 2024-04-28T12:43:51.403 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-28T12:43:51.487 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878/remote/smithi157/crash 2024-04-28T12:43:51.488 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-28T12:43:51.574 INFO:tasks.ceph:Shutting down logrotate 2024-04-28T12:43:51.574 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:43:51.582 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-28T12:43:51.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-28T12:43:51.785 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-28T12:43:51.875 INFO:tasks.ceph:Compressing logs... 2024-04-28T12:43:51.875 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-28T12:43:51.883 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-28T12:43:51.955 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40817.log 2024-04-28T12:43:51.956 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44814.log 2024-04-28T12:43:51.957 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106797.log 2024-04-28T12:43:51.958 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40817.log.gz 2024-04-28T12:43:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44898.log 2024-04-28T12:43:51.958 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44814.log.gz 2024-04-28T12:43:51.959 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106797.log: gzip -5 88.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.106797.log.gz 2024-04-28T12:43:51.959 INFO:teuthology.orchestra.run.smithi131.stderr: -- /var/log/ceph/ceph-client.admin.45843.log 2024-04-28T12:43:51.961 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42119.log 2024-04-28T12:43:51.962 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44898.log.gz 2024-04-28T12:43:51.962 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44940.log 88.7% 2024-04-28T12:43:51.962 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45843.log.gz 2024-04-28T12:43:51.966 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41279.log 2024-04-28T12:43:51.966 INFO:teuthology.orchestra.run.smithi131.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42119.log.gz 2024-04-28T12:43:51.966 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46479.log 2024-04-28T12:43:51.966 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44940.log.gz 2024-04-28T12:43:51.966 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106545.log 2024-04-28T12:43:51.967 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41279.log.gz 2024-04-28T12:43:51.967 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46479.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46479.log.gz 2024-04-28T12:43:51.967 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45885.log 2024-04-28T12:43:51.985 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33685.log 2024-04-28T12:43:51.985 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-28T12:43:51.985 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.33685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33485.log 2024-04-28T12:43:51.986 INFO:teuthology.orchestra.run.smithi157.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33685.log.gz 2024-04-28T12:43:51.986 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33565.log 2024-04-28T12:43:51.986 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.33485.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-28T12:43:51.986 INFO:teuthology.orchestra.run.smithi157.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33485.log.gz 2024-04-28T12:43:51.986 INFO:teuthology.orchestra.run.smithi157.stderr: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-28T12:43:51.986 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.33565.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-28T12:43:51.986 INFO:teuthology.orchestra.run.smithi157.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33565.log.gz 2024-04-28T12:43:51.986 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-28T12:43:51.986 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-28T12:43:51.989 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38251.log 2024-04-28T12:43:51.989 INFO:teuthology.orchestra.run.smithi157.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-28T12:43:51.990 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-28T12:43:51.990 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.38251.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-28T12:43:51.990 INFO:teuthology.orchestra.run.smithi157.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.1.38251.log.gz 2024-04-28T12:43:51.990 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-28T12:43:51.990 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33765.log 2024-04-28T12:43:51.990 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-28T12:43:51.990 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.33765.log: /var/log/ceph/ceph-osd.6.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33765.log.gz 2024-04-28T12:43:51.990 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33725.log 2024-04-28T12:43:51.996 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.33725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33645.log 2024-04-28T12:43:51.996 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.33645.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.33725.log.gz 2024-04-28T12:43:51.996 INFO:teuthology.orchestra.run.smithi157.stderr: -5 --verbose -- /var/log/ceph/ceph-client.3.38725.log 2024-04-28T12:43:51.996 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33605.log 2024-04-28T12:43:52.000 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48505.log 2024-04-28T12:43:52.000 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106545.log.gz 2024-04-28T12:43:52.000 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45885.log: gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.45885.log.gz 2024-04-28T12:43:52.000 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45471.log 2024-04-28T12:43:52.000 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40691.log 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48505.log.gz 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45161.log 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45471.log.gz 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47415.log 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40691.log.gz 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45161.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45161.log.gz 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48631.log 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45513.log 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47415.log.gz 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48673.log 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48631.log.gz 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45513.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45513.log.gz 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46521.log 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105518.log 2024-04-28T12:43:52.001 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48673.log.gz 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46521.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46521.log.gz 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42245.log 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42455.log 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105518.log.gz 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39347.log 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42245.log.gz 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42455.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42455.log.gzgzip 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43848.log 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46827.log 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39347.log.gz 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43848.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43848.log.gz 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47793.log 2024-04-28T12:43:52.002 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106713.log 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46827.log.gz 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47793.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47793.log.gz 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46011.log 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106713.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106713.log.gz 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106419.log 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46011.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46011.log.gz 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38837.log 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106419.log.gz 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45717.log 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38837.log: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38837.log.gz 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42371.log 2024-04-28T12:43:52.003 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43974.log 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45717.log.gz 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40243.log 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42371.log.gz 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48123.log 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43974.log.gz 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40243.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40243.log.gz 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41617.log 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43678.log 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48123.log.gz 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41617.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41617.log.gz 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48841.log 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46623.log 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43678.log.gz 2024-04-28T12:43:52.004 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48841.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48841.log.gz 2024-04-28T12:43:52.005 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-28T12:43:52.005 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43076.log 2024-04-28T12:43:52.005 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46623.log.gz 2024-04-28T12:43:52.010 INFO:teuthology.orchestra.run.smithi157.stderr: 88.5%/var/log/ceph/ceph-client.3.38725.log: /var/log/ceph/ceph-client.admin.33605.log: -- replaced with /var/log/ceph/ceph-client.admin.33645.log.gz 2024-04-28T12:43:52.010 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-28T12:43:52.011 INFO:teuthology.orchestra.run.smithi157.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33605.log.gz 2024-04-28T12:43:52.011 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-28T12:43:52.011 INFO:teuthology.orchestra.run.smithi157.stderr: 92.2%/var/log/ceph/ceph-mds.d.log: -- replaced with /var/log/ceph/ceph-client.3.38725.log.gz 2024-04-28T12:43:52.011 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33525.log 2024-04-28T12:43:52.011 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.33525.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33525.log.gz 2024-04-28T12:43:52.012 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45303.log 2024-04-28T12:43:52.012 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.43076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41867.log 2024-04-28T12:43:52.012 INFO:teuthology.orchestra.run.smithi131.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.43076.log.gz 2024-04-28T12:43:52.012 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49069.log 2024-04-28T12:43:52.012 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45303.log.gz 2024-04-28T12:43:52.012 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47163.log 2024-04-28T12:43:52.012 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41867.log.gz 2024-04-28T12:43:52.012 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43574.log 2024-04-28T12:43:52.013 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49069.log.gz 2024-04-28T12:43:52.013 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41614.log 2024-04-28T12:43:52.013 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47163.log.gz 2024-04-28T12:43:52.013 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.2.43574.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-28T12:43:52.013 INFO:teuthology.orchestra.run.smithi131.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.2.43574.log.gz 2024-04-28T12:43:52.013 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43180.log 2024-04-28T12:43:52.013 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41614.log.gz 2024-04-28T12:43:52.013 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44436.log 2024-04-28T12:43:52.013 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43180.log: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-28T12:43:52.013 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41069.log 2024-04-28T12:43:52.013 INFO:teuthology.orchestra.run.smithi131.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43180.log.gz 2024-04-28T12:43:52.015 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49027.log 2024-04-28T12:43:52.015 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44436.log.gz 2024-04-28T12:43:52.015 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105770.log 2024-04-28T12:43:52.015 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41069.log.gz 2024-04-28T12:43:52.015 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44562.log 2024-04-28T12:43:52.015 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49027.log.gz 2024-04-28T12:43:52.016 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46785.log 2024-04-28T12:43:52.016 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105770.log.gz 2024-04-28T12:43:52.016 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42539.log 2024-04-28T12:43:52.017 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44562.log.gz 2024-04-28T12:43:52.017 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40201.log 2024-04-28T12:43:52.018 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46785.log.gz 2024-04-28T12:43:52.018 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48333.log 2024-04-28T12:43:52.018 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42539.log.gz 2024-04-28T12:43:52.019 INFO:teuthology.orchestra.run.smithi157.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-28T12:43:52.020 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46211.log 2024-04-28T12:43:52.020 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40201.log.gz 2024-04-28T12:43:52.020 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44646.log 2024-04-28T12:43:52.021 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48333.log.gz 2024-04-28T12:43:52.021 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46211.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46211.log.gz 2024-04-28T12:43:52.021 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-28T12:43:52.022 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40285.log 2024-04-28T12:43:52.023 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44646.log.gz 2024-04-28T12:43:52.023 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48375.log 2024-04-28T12:43:52.024 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42203.log 2024-04-28T12:43:52.024 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40285.log.gz 2024-04-28T12:43:52.026 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44730.log 2024-04-28T12:43:52.026 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48375.log.gz 2024-04-28T12:43:52.027 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45591.log 2024-04-28T12:43:52.027 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42203.log.gz 2024-04-28T12:43:52.028 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47457.log 2024-04-28T12:43:52.028 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44730.log.gz 2024-04-28T12:43:52.029 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45591.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45591.log.gz 2024-04-28T12:43:52.029 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40117.log 2024-04-28T12:43:52.030 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38701.log 2024-04-28T12:43:52.030 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47457.log.gz 2024-04-28T12:43:52.031 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40117.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40117.log.gz 2024-04-28T12:43:52.031 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46053.log 2024-04-28T12:43:52.032 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45345.log 2024-04-28T12:43:52.033 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38701.log.gz 2024-04-28T12:43:52.033 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46053.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46053.log.gz 2024-04-28T12:43:52.034 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38745.log 2024-04-28T12:43:52.034 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47079.log 2024-04-28T12:43:52.035 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45345.log.gz 2024-04-28T12:43:52.036 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38745.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38745.log.gz 2024-04-28T12:43:52.036 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39179.log 2024-04-28T12:43:52.037 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48165.log 2024-04-28T12:43:52.037 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47079.log.gz 2024-04-28T12:43:52.038 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39179.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39179.log.gz 2024-04-28T12:43:52.038 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44226.log 2024-04-28T12:43:52.039 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42287.log 2024-04-28T12:43:52.039 INFO:teuthology.orchestra.run.smithi131.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48165.log.gz 2024-04-28T12:43:52.040 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44226.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44226.log.gz 2024-04-28T12:43:52.040 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105854.log 2024-04-28T12:43:52.041 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42287.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-28T12:43:52.042 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42287.log.gz 2024-04-28T12:43:52.042 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105854.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105854.log.gz 2024-04-28T12:43:52.042 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44058.log 2024-04-28T12:43:52.043 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107049.log 2024-04-28T12:43:52.044 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44058.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44058.log.gz 2024-04-28T12:43:52.045 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106881.log 2024-04-28T12:43:52.046 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47709.log 2024-04-28T12:43:52.046 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.107049.log: 92.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-28T12:43:52.047 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106881.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107049.log.gz 2024-04-28T12:43:52.047 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106881.log.gz 2024-04-28T12:43:52.047 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48901.log 2024-04-28T12:43:52.048 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46743.log 2024-04-28T12:43:52.049 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48901.log: gzip 89.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.47709.log.gz -- 2024-04-28T12:43:52.049 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.47995.log 2024-04-28T12:43:52.050 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48901.log.gz 2024-04-28T12:43:52.050 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46743.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46743.log.gz 2024-04-28T12:43:52.050 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106292.log 2024-04-28T12:43:52.051 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46337.log 2024-04-28T12:43:52.051 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47995.log.gz 2024-04-28T12:43:52.052 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106292.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.106292.log.gz -5 2024-04-28T12:43:52.052 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33678.log 2024-04-28T12:43:52.053 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44394.log 2024-04-28T12:43:52.053 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46337.log.gz 2024-04-28T12:43:52.054 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.33678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45387.log 2024-04-28T12:43:52.055 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33678.log.gz 2024-04-28T12:43:52.055 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48943.log 2024-04-28T12:43:52.056 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44394.log.gz 2024-04-28T12:43:52.057 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45387.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-osd.1.log -- replaced with /var/log/ceph/ceph-client.admin.45387.log.gz 2024-04-28T12:43:52.057 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-04-28T12:43:52.058 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48943.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-28T12:43:52.058 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48943.log.gz 2024-04-28T12:43:52.059 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106155.log 2024-04-28T12:43:52.060 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47751.log 2024-04-28T12:43:52.062 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47583.log 2024-04-28T12:43:52.062 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106155.log.gz 2024-04-28T12:43:52.063 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105350.log 2024-04-28T12:43:52.063 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47751.log.gz 2024-04-28T12:43:52.063 INFO:teuthology.orchestra.run.smithi157.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-28T12:43:52.064 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38628.log 2024-04-28T12:43:52.064 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47583.log.gz 2024-04-28T12:43:52.065 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40327.log 2024-04-28T12:43:52.066 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105350.log.gz 2024-04-28T12:43:52.067 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38628.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-28T12:43:52.067 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38628.log.gz 2024-04-28T12:43:52.068 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46869.log 2024-04-28T12:43:52.068 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40327.log.gz 2024-04-28T12:43:52.069 INFO:teuthology.orchestra.run.smithi131.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-28T12:43:52.069 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47915.log 2024-04-28T12:43:52.071 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42077.log 2024-04-28T12:43:52.071 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46869.log.gz 2024-04-28T12:43:52.072 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43932.log 2024-04-28T12:43:52.072 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47915.log.gz 2024-04-28T12:43:52.073 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49497.log 2024-04-28T12:43:52.074 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42077.log.gz 2024-04-28T12:43:52.074 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46953.log 2024-04-28T12:43:52.074 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43932.log.gz 2024-04-28T12:43:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.49497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41621.log 2024-04-28T12:43:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39455.log 2024-04-28T12:43:52.077 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46953.log.gz 2024-04-28T12:43:52.078 INFO:teuthology.orchestra.run.smithi157.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-28T12:43:52.078 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41290.log 2024-04-28T12:43:52.078 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41621.log.gz 2024-04-28T12:43:52.080 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48081.log 2024-04-28T12:43:52.080 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39455.log.gz 2024-04-28T12:43:52.081 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42035.log 2024-04-28T12:43:52.081 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41290.log.gz 2024-04-28T12:43:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106587.log 2024-04-28T12:43:52.082 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48081.log.gz 2024-04-28T12:43:52.083 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42035.log: 95.4% -- replaced with /var/log/ceph/ceph-client.0.49497.log.gz 2024-04-28T12:43:52.084 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42035.log.gz 2024-04-28T12:43:52.084 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41629.log 2024-04-28T12:43:52.085 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106587.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106587.log.gz 2024-04-28T12:43:52.085 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106461.log 2024-04-28T12:43:52.087 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46581.log 2024-04-28T12:43:52.087 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41629.log.gz 2024-04-28T12:43:52.088 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105392.log 2024-04-28T12:43:52.088 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106461.log.gz 2024-04-28T12:43:52.089 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38923.log 2024-04-28T12:43:52.089 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46581.log.gz 2024-04-28T12:43:52.091 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38879.log 2024-04-28T12:43:52.091 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105392.log.gz 2024-04-28T12:43:52.092 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40469.log 2024-04-28T12:43:52.092 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38923.log.gz 2024-04-28T12:43:52.093 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44856.log 2024-04-28T12:43:52.093 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38879.log.gz 2024-04-28T12:43:52.094 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38527.log 2024-04-28T12:43:52.095 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40469.log.gz 2024-04-28T12:43:52.095 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106755.log 2024-04-28T12:43:52.096 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44856.log.gz 2024-04-28T12:43:52.097 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45927.log 2024-04-28T12:43:52.097 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38527.log.gz 2024-04-28T12:43:52.098 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106240.log 2024-04-28T12:43:52.098 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106755.log.gz 2024-04-28T12:43:52.099 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44520.log 2024-04-28T12:43:52.099 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45927.log.gz 2024-04-28T12:43:52.101 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49111.log 2024-04-28T12:43:52.101 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106240.log.gz 2024-04-28T12:43:52.102 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46169.log 2024-04-28T12:43:52.102 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44520.log.gz 2024-04-28T12:43:52.103 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46995.log 2024-04-28T12:43:52.103 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49111.log.gz 2024-04-28T12:43:52.104 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49219.log 2024-04-28T12:43:52.105 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46169.log.gz 2024-04-28T12:43:52.106 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42667.log 2024-04-28T12:43:52.106 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46995.log.gz 2024-04-28T12:43:52.106 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48249.log 2024-04-28T12:43:52.107 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49219.log.gz 2024-04-28T12:43:52.108 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46701.log 2024-04-28T12:43:52.108 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42667.log.gz 2024-04-28T12:43:52.109 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47835.log 2024-04-28T12:43:52.109 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48249.log.gz 2024-04-28T12:43:52.110 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41291.log 2024-04-28T12:43:52.110 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46701.log.gz 2024-04-28T12:43:52.112 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48757.log 2024-04-28T12:43:52.112 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47835.log.gz 2024-04-28T12:43:52.113 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105476.log 2024-04-28T12:43:52.113 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41291.log.gz 2024-04-28T12:43:52.114 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41624.log 2024-04-28T12:43:52.114 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48757.log.gz 2024-04-28T12:43:52.115 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41627.log 2024-04-28T12:43:52.116 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105476.log.gz 2024-04-28T12:43:52.117 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41909.log 2024-04-28T12:43:52.117 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41624.log.gz 2024-04-28T12:43:52.118 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41627.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-28T12:43:52.118 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41627.log.gz 2024-04-28T12:43:52.119 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47667.log 2024-04-28T12:43:52.119 INFO:teuthology.orchestra.run.smithi131.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41909.log.gz 2024-04-28T12:43:52.121 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48461.log 2024-04-28T12:43:52.122 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40985.log 2024-04-28T12:43:52.122 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47667.log.gz 2024-04-28T12:43:52.123 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47037.log 2024-04-28T12:43:52.123 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48461.log.gz 2024-04-28T12:43:52.126 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40985.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40985.log.gz 2024-04-28T12:43:52.126 INFO:teuthology.orchestra.run.smithi131.stderr:gzip/var/log/ceph/ceph-client.admin.47037.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48417.log 2024-04-28T12:43:52.126 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47037.log.gz 2024-04-28T12:43:52.126 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48715.log 2024-04-28T12:43:52.128 INFO:teuthology.orchestra.run.smithi157.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-28T12:43:52.128 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47331.log 2024-04-28T12:43:52.128 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48417.log.gz 2024-04-28T12:43:52.130 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48715.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48715.log.gz 2024-04-28T12:43:52.130 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47247.log 2024-04-28T12:43:52.131 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47331.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47331.log.gz 2024-04-28T12:43:52.132 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40775.log 2024-04-28T12:43:52.133 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106503.log 2024-04-28T12:43:52.133 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47247.log.gz 2024-04-28T12:43:52.134 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40775.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40775.log.gz 2024-04-28T12:43:52.135 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40553.log 2024-04-28T12:43:52.135 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106503.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106503.log.gz 2024-04-28T12:43:52.136 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40385.log 2024-04-28T12:43:52.138 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106377.log 2024-04-28T12:43:52.138 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40553.log.gz 2024-04-28T12:43:52.139 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40385.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40385.log.gz 2024-04-28T12:43:52.140 INFO:teuthology.orchestra.run.smithi131.stderr:gzip/var/log/ceph/ceph-client.admin.106377.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41610.log 2024-04-28T12:43:52.140 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106377.log.gz 2024-04-28T12:43:52.141 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44688.log 2024-04-28T12:43:52.142 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39389.log 2024-04-28T12:43:52.142 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41610.log.gz 2024-04-28T12:43:52.144 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44688.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44688.log.gz 2024-04-28T12:43:52.144 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45429.log 2024-04-28T12:43:52.146 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33310.log 2024-04-28T12:43:52.146 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.39389.log.gz 2024-04-28T12:43:52.147 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106923.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45429.log.gz 2024-04-28T12:43:52.147 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-04-28T12:43:52.149 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.33310.log: gzip -5 --verbose 92.4% -- replaced with /var/log/ceph/ceph-client.admin.33310.log.gz 2024-04-28T12:43:52.149 INFO:teuthology.orchestra.run.smithi131.stderr: -- /var/log/ceph/ceph-client.admin.39009.log 2024-04-28T12:43:52.149 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106923.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106923.log.gz 2024-04-28T12:43:52.150 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105560.log 2024-04-28T12:43:52.151 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106965.log 2024-04-28T12:43:52.151 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39009.log.gz 2024-04-28T12:43:52.153 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44268.log 2024-04-28T12:43:52.153 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105560.log.gz 2024-04-28T12:43:52.154 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106965.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106965.log.gz 2024-04-28T12:43:52.155 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44268.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44268.log.gz 2024-04-28T12:43:52.155 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42329.log 2024-04-28T12:43:52.157 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45119.log 2024-04-28T12:43:52.159 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106629.log 2024-04-28T12:43:52.159 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42329.log.gz 2024-04-28T12:43:52.159 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45119.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45035.log 2024-04-28T12:43:52.160 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45119.log.gz 2024-04-28T12:43:52.161 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106629.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106629.log.gz 2024-04-28T12:43:52.161 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107009.log 2024-04-28T12:43:52.163 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45035.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45035.log.gz 2024-04-28T12:43:52.163 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48589.log 2024-04-28T12:43:52.164 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.107009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46295.log 2024-04-28T12:43:52.165 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107009.log.gz 2024-04-28T12:43:52.165 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48589.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.48589.log.gz --verbose 2024-04-28T12:43:52.166 INFO:teuthology.orchestra.run.smithi131.stderr: -- /var/log/ceph/ceph-client.admin.106335.log 2024-04-28T12:43:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106197.log 2024-04-28T12:43:52.167 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46295.log.gz 2024-04-28T12:43:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44142.log 2024-04-28T12:43:52.168 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106335.log.gz 2024-04-28T12:43:52.170 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41951.log 2024-04-28T12:43:52.170 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106197.log.gz 2024-04-28T12:43:52.171 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39263.log 2024-04-28T12:43:52.171 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44142.log.gz 2024-04-28T12:43:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48799.log 2024-04-28T12:43:52.172 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41951.log.gz 2024-04-28T12:43:52.173 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68489.log 2024-04-28T12:43:52.173 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39263.log.gz 2024-04-28T12:43:52.175 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33578.log 2024-04-28T12:43:52.175 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48799.log.gz 2024-04-28T12:43:52.176 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68489.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68489.log.gz 2024-04-28T12:43:52.176 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39137.log 2024-04-28T12:43:52.177 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.33578.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33578.log.gz 2024-04-28T12:43:52.178 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48291.log 2024-04-28T12:43:52.179 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38967.log 2024-04-28T12:43:52.179 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39137.log.gz 2024-04-28T12:43:52.180 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40901.log 2024-04-28T12:43:52.180 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48291.log.gz 2024-04-28T12:43:52.182 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38967.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.46395.log -- replaced with /var/log/ceph/ceph-client.admin.38967.log.gz 2024-04-28T12:43:52.182 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-04-28T12:43:52.183 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43720.log 2024-04-28T12:43:52.183 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40901.log.gz 2024-04-28T12:43:52.184 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45245.log 2024-04-28T12:43:52.184 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46395.log.gz 2024-04-28T12:43:52.186 INFO:teuthology.orchestra.run.smithi157.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-28T12:43:52.186 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46127.log 2024-04-28T12:43:52.186 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43720.log.gz 2024-04-28T12:43:52.187 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105602.log 2024-04-28T12:43:52.187 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45245.log.gz 2024-04-28T12:43:52.188 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106839.log 2024-04-28T12:43:52.189 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46127.log.gz 2024-04-28T12:43:52.190 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105602.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105602.log.gz 2024-04-28T12:43:52.194 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41293.log 2024-04-28T12:43:52.194 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39051.log 2024-04-28T12:43:52.195 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106839.log.gz 2024-04-28T12:43:52.195 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106671.log 2024-04-28T12:43:52.195 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41293.log.gz 2024-04-28T12:43:52.195 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48037.log 2024-04-28T12:43:52.195 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39051.log.gz 2024-04-28T12:43:52.196 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106671.log: gzip -5 --verbose 88.1% -- /var/log/ceph/ceph-client.admin.105896.log 2024-04-28T12:43:52.196 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106671.log.gz 2024-04-28T12:43:52.197 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48037.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48037.log.gz 2024-04-28T12:43:52.198 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47625.log 2024-04-28T12:43:52.199 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105896.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27296.log 2024-04-28T12:43:52.199 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105896.log.gz 2024-04-28T12:43:52.200 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45203.log 2024-04-28T12:43:52.200 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47625.log.gz 2024-04-28T12:43:52.201 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.tmp-client.admin.27296.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27296.log.gz 2024-04-28T12:43:52.202 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67975.log 2024-04-28T12:43:52.203 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44352.log 2024-04-28T12:43:52.203 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45203.log.gz 2024-04-28T12:43:52.204 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48985.log 2024-04-28T12:43:52.204 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67975.log.gz 2024-04-28T12:43:52.206 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39221.log 2024-04-28T12:43:52.206 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44352.log.gz 2024-04-28T12:43:52.207 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48985.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48985.log.gz 2024-04-28T12:43:52.207 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44184.log 2024-04-28T12:43:52.208 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39221.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38795.log 2024-04-28T12:43:52.208 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39221.log.gz 2024-04-28T12:43:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105434.log 2024-04-28T12:43:52.210 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44184.log.gz 2024-04-28T12:43:52.211 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38795.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38795.log.gz 2024-04-28T12:43:52.212 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45759.log 2024-04-28T12:43:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105434.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105434.log.gz 2024-04-28T12:43:52.213 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44100.log 2024-04-28T12:43:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45759.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45759.log.gz 2024-04-28T12:43:52.215 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47289.log 2024-04-28T12:43:52.216 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41280.log 2024-04-28T12:43:52.216 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44100.log.gz 2024-04-28T12:43:52.217 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47289.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47289.log.gz 2024-04-28T12:43:52.218 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46437.log 2024-04-28T12:43:52.218 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41280.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41280.log.gz 2024-04-28T12:43:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38431.log 2024-04-28T12:43:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105686.log 2024-04-28T12:43:52.220 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46437.log.gz 2024-04-28T12:43:52.221 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38431.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38431.log.gz 2024-04-28T12:43:52.222 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47373.log 2024-04-28T12:43:52.223 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105686.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43890.log 2024-04-28T12:43:52.223 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.105686.log.gz 2024-04-28T12:43:52.224 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47205.log 2024-04-28T12:43:52.224 INFO:teuthology.orchestra.run.smithi131.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47373.log.gz 2024-04-28T12:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43890.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43890.log.gz 2024-04-28T12:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45969.log 2024-04-28T12:43:52.227 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47205.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47205.log.gz 2024-04-28T12:43:52.227 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105644.log 2024-04-28T12:43:52.229 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39892.log 2024-04-28T12:43:52.229 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45969.log.gz 2024-04-28T12:43:52.230 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105644.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105644.log.gz 2024-04-28T12:43:52.231 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44772.log 2024-04-28T12:43:52.232 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41292.log 2024-04-28T12:43:52.233 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44772.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.39892.log.gz 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44772.log.gz 2024-04-28T12:43:52.234 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-04-28T12:43:52.234 INFO:teuthology.orchestra.run.smithi131.stderr:gzip/var/log/ceph/ceph-client.admin.41292.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.106113.log 2024-04-28T12:43:52.235 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41292.log.gz 2024-04-28T12:43:52.235 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40427.log 2024-04-28T12:43:52.237 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.106113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44478.log 2024-04-28T12:43:52.237 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106113.log.gz 2024-04-28T12:43:52.238 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40427.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40427.log.gz 2024-04-28T12:43:52.239 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41027.log 2024-04-28T12:43:52.240 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41111.log 2024-04-28T12:43:52.240 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44478.log.gz 2024-04-28T12:43:52.241 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40511.log 2024-04-28T12:43:52.242 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41027.log.gz 2024-04-28T12:43:52.242 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41111.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41111.log.gz 2024-04-28T12:43:52.243 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40511.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40511.log.gz 2024-04-28T12:43:52.244 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39305.log 2024-04-28T12:43:52.245 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46911.log 2024-04-28T12:43:52.247 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105308.log 2024-04-28T12:43:52.247 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39305.log.gz 2024-04-28T12:43:52.248 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46911.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46911.log.gz 2024-04-28T12:43:52.248 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47499.log 2024-04-28T12:43:52.250 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105308.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-28T12:43:52.250 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105308.log.gz 2024-04-28T12:43:52.251 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48207.log 2024-04-28T12:43:52.251 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47499.log.gz 2024-04-28T12:43:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.48207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42497.log 2024-04-28T12:43:52.254 INFO:teuthology.orchestra.run.smithi131.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48207.log.gz 2024-04-28T12:43:52.254 INFO:teuthology.orchestra.run.smithi131.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-28T12:43:52.255 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41825.log 2024-04-28T12:43:52.256 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42161.log 2024-04-28T12:43:52.256 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42497.log.gz 2024-04-28T12:43:52.257 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49153.log 2024-04-28T12:43:52.257 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41825.log.gz 2024-04-28T12:43:52.259 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42161.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42161.log.gz 2024-04-28T12:43:52.259 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105812.log 2024-04-28T12:43:52.261 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49153.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.105812.log: -- /var/log/ceph/ceph-client.admin.47541.log 2024-04-28T12:43:52.262 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49153.log.gz 2024-04-28T12:43:52.262 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105812.log.gz 2024-04-28T12:43:52.263 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40943.log 2024-04-28T12:43:52.264 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46253.log 2024-04-28T12:43:52.264 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47541.log.gz 2024-04-28T12:43:52.265 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45675.log 2024-04-28T12:43:52.265 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40943.log.gz 2024-04-28T12:43:52.266 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46253.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46253.log.gz 2024-04-28T12:43:52.267 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44310.log 2024-04-28T12:43:52.268 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45675.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45675.log.gz -5 2024-04-28T12:43:52.268 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40595.log 2024-04-28T12:43:52.269 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105728.log 2024-04-28T12:43:52.269 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44310.log.gz 2024-04-28T12:43:52.271 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40595.log: gzip -5 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40595.log.gz 2024-04-28T12:43:52.271 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44604.log 2024-04-28T12:43:52.272 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105728.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105728.log.gz 2024-04-28T12:43:52.272 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107091.log 2024-04-28T12:43:52.274 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44604.log: gzip 89.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.44604.log.gz -- 2024-04-28T12:43:52.274 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.45801.log 2024-04-28T12:43:52.275 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.107091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41628.log 2024-04-28T12:43:52.275 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107091.log.gz 2024-04-28T12:43:52.277 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41993.log 2024-04-28T12:43:52.277 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45801.log.gz 2024-04-28T12:43:52.278 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41628.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41628.log.gz 2024-04-28T12:43:52.279 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41993.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41993.log.gz 2024-04-28T12:43:52.280 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47121.log 2024-04-28T12:43:52.281 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45077.log 2024-04-28T12:43:52.282 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45633.log 2024-04-28T12:43:52.282 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47121.log.gz 2024-04-28T12:43:52.284 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41287.log 2024-04-28T12:43:52.284 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45077.log.gz 2024-04-28T12:43:52.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40733.log 2024-04-28T12:43:52.285 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45633.log.gz 2024-04-28T12:43:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41287.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41287.log.gz 2024-04-28T12:43:52.287 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105938.log 2024-04-28T12:43:52.288 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40733.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42413.log 2024-04-28T12:43:52.288 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40733.log.gz 2024-04-28T12:43:52.289 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.105938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41283.log 2024-04-28T12:43:52.289 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105938.log.gz 2024-04-28T12:43:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42413.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42413.log.gz 2024-04-28T12:43:52.291 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44016.log 2024-04-28T12:43:52.292 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48547.log 2024-04-28T12:43:52.292 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41283.log.gz 2024-04-28T12:43:52.294 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44016.log: gzip -5 --verbose -- 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gz 2024-04-28T12:43:52.294 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.40159.log 2024-04-28T12:43:52.295 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48547.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39095.log 2024-04-28T12:43:52.295 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48547.log.gz 2024-04-28T12:43:52.296 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40859.log 2024-04-28T12:43:52.297 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40159.log.gz 2024-04-28T12:43:52.298 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39095.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39095.log.gz 2024-04-28T12:43:52.299 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40859.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40859.log.gz 2024-04-28T12:43:52.587 INFO:teuthology.orchestra.run.smithi131.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-28T12:43:52.762 INFO:teuthology.orchestra.run.smithi131.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-28T12:43:52.789 INFO:teuthology.orchestra.run.smithi157.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-28T12:43:52.815 INFO:teuthology.orchestra.run.smithi157.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-28T12:43:52.835 INFO:teuthology.orchestra.run.smithi157.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-28T12:43:52.843 INFO:teuthology.orchestra.run.smithi131.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-28T12:43:52.845 INFO:teuthology.orchestra.run.smithi131.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-28T12:43:52.848 INFO:teuthology.orchestra.run.smithi157.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-28T12:43:52.851 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-04-28T12:43:52.851 INFO:teuthology.orchestra.run.smithi157.stderr:real 0m0.917s 2024-04-28T12:43:52.851 INFO:teuthology.orchestra.run.smithi157.stderr:user 0m3.621s 2024-04-28T12:43:52.851 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m0.392s 2024-04-28T12:43:52.997 INFO:teuthology.orchestra.run.smithi131.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-28T12:43:53.002 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-04-28T12:43:53.002 INFO:teuthology.orchestra.run.smithi131.stderr:real 0m1.074s 2024-04-28T12:43:53.002 INFO:teuthology.orchestra.run.smithi131.stderr:user 0m3.606s 2024-04-28T12:43:53.002 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m1.027s 2024-04-28T12:43:53.003 INFO:tasks.ceph:Archiving logs... 2024-04-28T12:43:53.003 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878/remote/smithi131/log 2024-04-28T12:43:53.004 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-28T12:43:53.753 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878/remote/smithi157/log 2024-04-28T12:43:53.754 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-28T12:43:54.321 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-28T12:43:54.331 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-28T12:43:54.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-28T12:43:54.339 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-28T12:43:54.480 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-28T12:43:54.489 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-28T12:43:54.489 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T12:43:54.497 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T12:43:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-04-28T12:43:54.551 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-04-28T12:43:54.555 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T12:43:54.555 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-04-28T12:43:54.555 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv01.front.sepia.ceph.com 3 6 377 50 +1853us[+1850us] +/- 60ms 2024-04-28T12:43:54.555 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -1186us[-1189us] +/- 78ms 2024-04-28T12:43:54.555 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -1202us[-1205us] +/- 73ms 2024-04-28T12:43:54.555 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:43:54.560 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T12:43:54.560 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-04-28T12:43:54.560 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +1830us[+1872us] +/- 60ms 2024-04-28T12:43:54.560 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 -1174us[-1174us] +/- 78ms 2024-04-28T12:43:54.561 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 -1215us[-1215us] +/- 73ms 2024-04-28T12:43:54.561 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:43:54.562 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-28T12:43:54.572 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-28T12:43:54.572 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-28T12:43:54.581 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-28T12:43:54.780 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-28T12:43:54.970 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T12:43:55.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:43:55.064 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T12:43:55.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:43:55.155 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-28T12:43:55.165 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-28T12:43:55.174 INFO:teuthology.task.internal:Duration was 1017.608553 seconds 2024-04-28T12:43:55.175 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-28T12:43:55.184 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-28T12:43:55.184 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-28T12:43:55.192 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-28T12:43:55.316 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T12:43:55.329 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T12:43:55.591 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-28T12:43:55.592 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-04-28T12:43:55.592 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-28T12:43:55.665 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-04-28T12:43:55.665 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-28T12:43:55.737 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-28T12:43:55.737 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-28T12:43:55.745 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-28T12:43:55.829 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-28T12:43:55.829 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-28T12:43:55.834 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-28T12:43:56.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-28T12:43:56.015 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-28T12:43:56.016 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-28T12:43:56.098 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-28T12:43:56.186 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-28T12:43:56.197 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-28T12:43:56.205 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-28T12:43:56.300 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-04-28T12:43:56.301 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-04-28T12:43:56.342 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-28T12:43:56.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:43:56.388 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-28T12:43:56.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:43:56.444 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-28T12:43:56.455 INFO:teuthology.task.internal:Transferring archived files... 2024-04-28T12:43:56.455 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878/remote/smithi131 2024-04-28T12:43:56.456 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-28T12:43:56.552 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878/remote/smithi157 2024-04-28T12:43:56.552 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-28T12:43:56.648 INFO:teuthology.task.internal:Removing archive directory... 2024-04-28T12:43:56.648 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-28T12:43:56.656 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-28T12:43:56.709 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-28T12:43:56.719 INFO:teuthology.task.internal:Not uploading archives. 2024-04-28T12:43:56.719 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-28T12:43:56.728 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-28T12:43:56.729 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-28T12:43:56.734 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-28T12:43:56.780 INFO:teuthology.orchestra.run.smithi131.stdout: 393229 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 28 12:43 /home/ubuntu/cephtest 2024-04-28T12:43:56.780 INFO:teuthology.orchestra.run.smithi131.stdout: 527347 4 d--------- 2 ubuntu ubuntu 4096 Apr 28 12:39 /home/ubuntu/cephtest/mnt.0 2024-04-28T12:43:56.780 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-28T12:43:56.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:43:56.783 INFO:teuthology.orchestra.run.smithi131.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-28T12:43:56.783 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi131 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-28T12:43:56.784 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-28T12:43:56.794 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-28T12:43:56.805 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi131 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-28T12:43:56.805 INFO:teuthology.nuke:Checking targets against current locks 2024-04-28T12:43:56.809 INFO:teuthology.orchestra.run.smithi157.stdout: 393242 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 28 12:43 /home/ubuntu/cephtest 2024-04-28T12:43:56.832 DEBUG:teuthology.nuke:shortname: smithi131 2024-04-28T12:43:56.833 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-28T12:43:56.855 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 12:12:14.727757', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNGKrQx3beiFGGGKr1IgBUo3gSRENd3cGQgZpEJ851txl5x6woWTKky3Fo7G49I7uRXQviEpZe6QxuTI98CKcvw='} 2024-04-28T12:43:56.858 DEBUG:teuthology.nuke:shortname: smithi157 2024-04-28T12:43:56.858 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-28T12:43:56.875 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 12:12:14.725859', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAKcrhOvz6kpvYZgifp4WCPpwgGaA25tfLv08kMoYPpogHbOyuqFYVN/ZfdWJX+EUTPcB4G/dIKH8aVQiM70axQ='} 2024-04-28T12:43:56.900 INFO:teuthology.orchestra.console.smithi131:Power off 2024-04-28T12:43:56.900 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-28T12:43:56.928 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off 2024-04-28T12:43:56.928 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T12:43:56.943 INFO:teuthology.orchestra.console.smithi157:Power off 2024-04-28T12:43:56.943 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-28T12:43:56.952 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2024-04-28T12:43:56.964 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2024-04-28T12:43:56.964 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T12:43:56.988 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-04-28T12:44:00.953 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T12:44:00.989 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T12:44:01.064 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2024-04-28T12:44:01.099 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-04-28T12:44:05.065 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T12:44:05.099 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T12:44:05.178 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2024-04-28T12:44:05.208 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2024-04-28T12:44:05.279 INFO:teuthology.orchestra.console.smithi131:Power off completed 2024-04-28T12:44:05.308 INFO:teuthology.orchestra.console.smithi157:Power off completed 2024-04-28T12:44:05.413 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} duration: 1017.6085531711578 failure_reason: 'Command failed on smithi131 with status 1: ''find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest''' flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology status: fail success: false 2024-04-28T12:44:05.413 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-28T12:44:05.490 INFO:teuthology.run:FAIL